Papers with neural models
Joint models for NLP (D18-3)
Copied to clipboard
| Challenge: | This tutorial reviews main approaches to joint modeling for statistical and neural methods. |
| Approach: | This tutorial reviews main approaches to joint modeling for both statistical and neural methods. |
| Outcome: | This tutorial reviews main approaches to joint modeling for statistical and neural methods. |
Unsupervised Natural Language Parsing (Introductory Tutorial) (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Unsupervised parsing learns a syntactic parser from training sentences without parse tree annotations. |
| Approach: | This tutorial will introduce what unsupervised parsing does and how it can be useful for and beyond syntactic parse. |
| Outcome: | This paper will provide an overview of major approaches to unsupervised parsing and analyze their strengths and weaknesses. |
IntelliCAT: Intelligent Machine Translation Post-Editing with Quality Estimation and Translation Suggestion (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing computer-aided translation tools require the translator to edit incorrect parts of a document, while ITP tools require fewer edits. |
| Approach: | They propose an interactive translation interface with neural models that streamline the post-editing process on machine translation output. |
| Outcome: | The proposed interface can significantly improve translation quality and a user study shows that it speeds up the post-editing process by 52.9% compared to translating from scratch. |
Data-Efficient Paraphrase Generation to Bootstrap Intent Classification and Slot Labeling for New Features in Task-Oriented Dialog Systems (2020.coling-industry)
Copied to clipboard
| Challenge: | a number of dialog systems have been developed to perform tasks with high accuracy on benchmarks, but there is a problem with annotated seed data. |
| Approach: | They propose a model that augments initial seed data by paraphrasing existing utterances automatically. |
| Outcome: | The proposed approach improves intent classification and slot labeling on a public dataset and with a real-world dialog system. |
Deep Learning Approaches to Text Production (N18-6)
Copied to clipboard
| Challenge: | Text production is a key component of many NLP applications . Claire Gardent is based in France and is pursuing research in text production . |
| Approach: | This tutorial will cover the fundamentals and state-of-the-art research on neural models for text production. |
| Outcome: | This tutorial will cover the fundamentals and the state-of-the-art research on neural models for text production. |
Neural Self-Training through Spaced Repetition (N19-1)
Copied to clipboard
| Challenge: | Existing methods for self-training rely on predetermined policies to sample unlabeled data. |
| Approach: | They propose a semi-supervised learning approach that uses spaced repetition to dynamically sample informative and diverse unlabeled instances with respect to individual learner and instance characteristics. |
| Outcome: | The proposed model outperforms existing semi-supervised learning approaches on publicly-available datasets. |
Neural Speech Translation using Lattice Transformations and Graph Networks (D19-53)
Copied to clipboard
| Challenge: | Existing work on end-to-end systems bypass the need for intermediate representations, but this approach is limited in practical applications. |
| Approach: | They propose a lattice-tosequence model which uses lattics as encoders and graph networks to address two problems by applying latticae transformations and a neural model. |
| Outcome: | The proposed model beats pipeline approaches while being orders of magnitude faster than previous work. |
Learning with Limited Text Data (2022.acl-tutorials)
Copied to clipboard
| Challenge: | Natural Language Processing (NLP) relies on labeled data to perform state-of-the-art performance . labeles are often required to label large amounts of textual data . this tutorial will provide an overview of labeleing in NLP . |
| Approach: | This tutorial will provide a systematic overview of methods for learning from limited labeled data. |
| Outcome: | This tutorial will provide a systematic and up-to-date overview of the proposed methods . it will highlight current challenges and future directions . |
PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation (2023.eacl-main)
Copied to clipboard
| Challenge: | Curriculum Data Augmentation (CDA) presents synthetic data with increasing difficulties to neural models. |
| Approach: | They propose a curriculum-aware paraphrase generation module with bottom-k sampling and cyclic learning strategy that passes through the curriculums multiple times. |
| Outcome: | The proposed framework surpasses competitive baselines on few-shot text classification and dialogue generation. |
Recovering Lexically and Semantically Reused Texts (2021.starsem-1)
Copied to clipboard
| Challenge: | Writers often repurpose material from existing texts when composing new documents. |
| Approach: | They propose to use local text reuse detection to detect localized regions of lexically or semantically similar text embedded in otherwise unrelated material. |
| Outcome: | The proposed methods perform better on three LTRD tasks, detecting plagiarism, modeling journalists’ use of press releases, and identifying scientists’ citation of earlier papers. |
Table Question Answering for Low-resourced Indic Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | TableQA is the task of answering questions over tables of structured information, returning individual cells or tables as output. |
| Approach: | They propose a fully automatic large-scale tableQA data generation process for low-resource languages with limited budget. |
| Outcome: | The proposed method outperforms state-of-the-art LLMs on two Indic languages with no tableQA datasets and models on different aspects including mathematical reasoning capabilities and zero-shot cross-lingual transfer. |
Methods for Estimating and Improving Robustness of Language Models (2022.naacl-srw)
Copied to clipboard
| Challenge: | Large language models suffer from weak generalisation ability due to shallow textual relations over full semantic complexity of the problem. |
| Approach: | They propose to incorporate some of these measures into training objectives to enhance distributional robustness of LLMs. |
| Outcome: | The proposed models outperform human models on complex tasks and outperformed other models on deep networks. |
Deep Learning for Dialogue Systems (C18-3)
Copied to clipboard
| Challenge: | Using deep learning to build robust and scalable spoken dialogue systems is still a challenging task. |
| Approach: | tutorial focuses on an overview of dialogue system development . goal-oriented spoken dialogue systems are most prominent component in virtual personal assistants . |
| Outcome: | This tutorial focuses on an overview of dialogue system development while summarizing the challenges. |
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2021.tacl-1)
Copied to clipboard
| Challenge: | Amnesic probing is a method that focuses on how information is being used, rather than on what information is encoded. |
| Approach: | They propose a method that focuses on how the information is being used rather than on what information is encoded. |
| Outcome: | The proposed method is based on a BERT dataset to ask questions that were not possible before . it shows that probing performance is not correlated to task importance . |
Neural Lexicons for Slot Tagging in Spoken Language Understanding (N19-2)
Copied to clipboard
| Challenge: | lexicons or gazettes are used to improve slot tagging in spoken language understanding systems. |
| Approach: | They develop models that encode lexicon information as neural features for use in a long-short term memory neural network. |
| Outcome: | The proposed models improve slot tagging with lexicons and gazettes . the results could be used to improve other natural language applications . |
Combine to Describe: Evaluating Compositional Generalization in Image Captioning (2022.acl-srw)
Copied to clipboard
| Challenge: | Recent work on compositionality has focused on the ability to combine simpler concepts to understand & generate arbitrarily more complex conceptual structures. |
| Approach: | They propose to use a set of image captioning models to benchmark their compositional generalization properties. |
| Outcome: | The proposed models do not generalize in terms of systematicity and productivity, but are robust to synonym substitutions. |
Youling: an AI-assisted Lyrics Creation System (2020.emnlp-demos)
Copied to clipboard
Rongsheng Zhang, Xiaoxi Mao, Le Li, Lin Jiang, Lin Chen, Zhiwei Hu, Yadong Xi, Changjie Fan, Minlie Huang
| Challenge: | Recent studies have focused on a single pass of lyrics generation with little human intervention. |
| Approach: | They propose an AI-assisted lyrics creation system that supports one pass full-text generation and interactive generation modes. |
| Outcome: | The proposed system supports full-text generation and interactive generation modes . it also provides a revision module which enables users to revise undesired lyrics repeatedly. |
NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit (P19-3)
Copied to clipboard
| Challenge: | NeuralClassifier is a toolkit for hierarchical multi-label text classification. |
| Approach: | They propose a toolkit for neural hierarchical multi-label text classification . they use a variety of text encoders to implement the model . |
| Outcome: | The proposed model achieves comparable performance with reported results in the literature. |
A Visuospatial Dataset for Naturalistic Verb Learning (2020.starsem-1)
Copied to clipboard
| Challenge: | a new dataset is available for training and evaluating grounded language models . our data is designed to emulate the quality of language data a pre-verbal child would have access to . |
| Approach: | They propose a dataset for training and evaluating grounded language models . they use naturalistic, spontaneous speech paired with richly grounded visuospatial context . |
| Outcome: | The proposed dataset compares two distributional semantics models with one that does not. |
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions (2024.naacl-demo)
Copied to clipboard
Zhengxuan Wu, Atticus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah Goodman, Christopher Manning, Christopher Potts
| Challenge: | Existing libraries are often project-based, but pyvene provides a unified and extensible framework for performing interventions on neural models and sharing the intervened upon models with others. |
| Approach: | They propose an open-source Python library that supports customizable interventions on a range of different PyTorch modules. |
| Outcome: | The proposed framework provides a unified and extensible framework for performing interventions on neural models and sharing the intervened upon models with others. |
Bend but Don’t Break? Multi-Challenge Stress Test for QA Models (D19-58)
Copied to clipboard
| Challenge: | a gap remains in reasoning ability compared to a human, and performance tends to degrade when models are exposed to less-constrained tasks. |
| Approach: | They conduct extensive qualitative and quantitative analyses on the results of four models across four datasets . they relate common errors to model capabilities and discuss a way forward . |
| Outcome: | The proposed model performance is based on the results of four models across four datasets. |
SummVis: Interactive Visual Analysis of Models, Data, and Evaluation for Text Summarization (2021.acl-demo)
Copied to clipboard
| Challenge: | despite advances in abstractive text summarization, the true performance and failure modes of modern neural models are not yet fully understood due to the black-box nature of neural models and unmanageable scale of recent datasets for manual analysis. |
| Approach: | They propose an open-source tool for visualizing abstractive summaries that enables fine-grained analysis of models, data, and evaluation metrics associated with text summarization. |
| Outcome: | The proposed tool can identify the shortcomings and failure modes of state-of-the-art summarization models. |
The Effects of Language Token Prefixing for Multilingual Machine Translation (2022.aacl-short)
Copied to clipboard
| Challenge: | In recent years, the field has moved towards large neural models either translating from or into many languages. |
| Approach: | They propose to prefix language tokens onto a source or target sequence to improve translation performance. |
| Outcome: | The proposed methods improve translation performance and source side prefixes improve translation. |
NeuSpell: A Neural Spelling Correction Toolkit (2020.emnlp-demos)
Copied to clipboard
| Challenge: | a new spelling correction toolkit is available for free. |
| Approach: | They propose an open-source toolkit for spelling correction in English . they train neural models using spelling errors in context and using richer contextual representations. |
| Outcome: | The proposed spell-checker improves accuracy on synthetic examples and richer representations of the context. |
Does Character-level Information Always Improve DRS-based Semantic Parsing? (2023.starsem-1)
Copied to clipboard
| Challenge: | incorporating character-level information does not improve the performance in English and German, and is not sensitive to correct character order in Dutch. |
| Approach: | They propose to incorporate character-level representations into a neural semantic parser for Discourse Representation Structures and to test their performance using order of character sequences. |
| Outcome: | The proposed parser improves in English, German, Dutch, and Italian in four languages. |
Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies show that high-quality rule sets struggle with high coverage. |
| Approach: | They propose three simple augmentations to existing rule sets to improve results . they propose transforming rules to their abductive forms and generating equivalent rules that use inverse forms of constituent relations . |
| Outcome: | The proposed methods achieve up to 7.1 pt MRR and 8.5 pT Hits@1 gains over using rules without augmentations. |
Nakdan: Professional Hebrew Diacritizer (2020.acl-demos)
Copied to clipboard
| Challenge: | a system for automatic diacritization of Hebrew Text is available for both casual and expert users. |
| Approach: | They propose a system for automatic diacritization of Hebrew Text . the system combines declarative linguistic knowledge with machine learning models . |
| Outcome: | The proposed system is available for both casual and expert users. |
Improving Chinese Story Generation via Awareness of Syntactic Dependencies and Semantics (2022.aacl-short)
Copied to clipboard
| Challenge: | Current neural models for Chinese story generation struggle to generate high-quality long text narratives due to ambiguity in syntactically parsing the Chinese language. |
| Approach: | They propose a framework that enhances the feature capturing mechanism by informing the generation model of dependencies between words and additionally augmenting the semantic representation learning through synonym denoising training. |
| Outcome: | The proposed framework outperforms the state-of-the-art Chinese generation models on all evaluation metrics, showing that it enhances dependency and semantic representation learning. |
Recipes for Building an Open-Domain Chatbot (2021.eacl-main)
Copied to clipboard
Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, Jason Weston
| Challenge: | Existing work shows that scaling models in the number of parameters and the size of the data they are trained on gives improved results, but other factors are important. |
| Approach: | They propose to build open-domain chatbots that can be scaled to improve their performance . they use a blend of cognitive and cognitive skills to build a model that combines these skills . |
| Outcome: | The proposed models outperform existing approaches in multi-turn dialogue on engagingness and humanness measurements. |
Character-Based Neural Networks for Sentence Pair Modeling (N18-2)
Copied to clipboard
| Challenge: | Sentence pair modeling is critical for many NLP tasks, such as paraphrase identification and semantic textual similarity. |
| Approach: | They propose to use subwords to represent sentences without pretrained word embeddings . they find that subword models can achieve new state-of-the-art results without pretraining . |
| Outcome: | The proposed models can achieve state-of-the-art results on two social media datasets and competitive results on news data for paraphrase identification. |
OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (D19-3)
Copied to clipboard
| Challenge: | OpenNRE provides a framework to implement neural relation extraction (RE) . the toolkit provides various functional modules based on TensorFlow and PyTorch . |
| Approach: | OpenNRE is an open-source framework to implement neural relation extraction models. they also release an online system to meet real-time extraction without any training and deployment. |
| Outcome: | OpenNRE provides a framework to implement neural models for relation extraction (RE) the toolkit also includes an online system to meet real-time extraction without training and deployment . |
Predicting pragmatic discourse features in the language of adults with autism spectrum disorder (2021.acl-srw)
Copied to clipboard
| Challenge: | Existing tools to quantify atypicality in discourse and pragmatics are difficult to precisely identify and quantify. |
| Approach: | They present a corpus of transcribed natural conversations produced in an experimental setting and annotate them for three pragmatic features on a three-point scale. |
| Outcome: | The proposed model yields higher accuracy than previous approaches for deriving these features, with F1 exceeding 0.82 for all three pragmatic features. |
Naver Labs Europe’s Systems for the Document-Level Generation and Translation Task at WNGT 2019 (D19-56)
Copied to clipboard
| Challenge: | Recent advances in machine translation and natural language generation have created many challenges in this field especially when context is considered. |
| Approach: | They propose to leverage data from machine translation and natural language generation tasks to do transfer learning between MT, NLG and MT with source-side metadata. |
| Outcome: | The proposed approach outperforms the previous state-of-the-art on the Rotowire NLG task. |
Coherent and Concise Radiology Report Generation via Context Specific Image Representations and Orthogonal Sentence States (2021.naacl-industry)
Copied to clipboard
| Challenge: | Neural models for text generation are often designed in an end-to-end fashion, limiting their practical usability in downstream applications. |
| Approach: | They propose a method to compute image representations specific to each sentential context and exploiting diverse sentence states to ensure topical continuity and content diversity of generated radiology reports. |
| Outcome: | The proposed method outperforms baselines on objective metrics and human evaluations by 18% and 29% respectively in the evaluation for informativeness and content ordering respectively. |
Neural Approaches for Natural Language Interfaces to Databases: A Survey (2020.coling-main)
Copied to clipboard
Radu Cristian Alexandru Iacob, Florin Brad, Elena-Simona Apostol, Ciprian-Octavian Truică, Ionel Alexandru Hosu, Traian Rebedea
| Challenge: | Interest in NLIDBs has resurged in the past years due to the availability of large datasets and improvements to neural sequence-to-sequence models. |
| Approach: | They focus on key design decisions behind current state of the art neural approaches . they highlight linking question tokens to database schema elements . |
| Outcome: | The proposed approaches are grouped into encoder and decoder improvements . they include better architectures for encoding the textual query taking into account the schema and improved generation of structured queries using autoregressive neural models. |
Text Generation from Discourse Representation Structures (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs) . |
| Approach: | They propose neural models to generate text from formal meaning representations based on Discourse Representation Structures (DRSs). |
| Outcome: | The proposed model achieves competitive performance on the GMB benchmark against several strong baselines. |
Document Summarization with Latent Queries (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing benchmarks for query-focused summarization are small for training large neural models. |
| Approach: | They propose a unified modeling framework for query-focused summarization . they model queries as discrete latent variables over document tokens . |
| Outcome: | The proposed framework outperforms strong comparison systems across benchmarks, query types, document settings, and target domains. |
Beyond Reptile: Meta-Learned Dot-Product Maximization between Gradients for Improved Single-Task Regularization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to improve generalization of neural models use a small component of the gradient for maximizing dot-product between batches. |
| Approach: | They propose to use a finite differences first-order algorithm to calculate a gradient from dot-product of gradients and regularize it. |
| Outcome: | The proposed method outperforms previous approaches of Reptile and MAML when used as a regularization technique. |
Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent graph-to-text models generate text from graph data using global or local aggregation . global node encoding allows explicit communication between two distant nodes, but fails to capture long-range relationships. |
| Approach: | They propose to combine global and local aggregation to learn node representations . they propose to use global and locally encoding to learn contextualized node embeddings based on graph data . |
| Outcome: | The proposed models outperform state-of-the-art models on two graph-to-text datasets by 18.01 and 63.69 points. |
Generating Equation by Utilizing Operators : GEO model (2020.coling-main)
Copied to clipboard
| Challenge: | Existing neural models that use hand-crafted features are expensive and lack domain-specific knowledge. |
| Approach: | They propose a GEO model that uses operator-based features to generate equations using natural language sentences. |
| Outcome: | The proposed model outperforms state-of-the-art models on two datasets and 82.1% in ALG514. |
Examining the Inductive Bias of Neural Language Models with Artificial Languages (2021.acl-long)
Copied to clipboard
| Challenge: | a novel method for investigating inductive biases of language models using artificial languages is proposed . we show that modern neural architectures used for language modeling are intrinsically black boxes . |
| Approach: | They propose a method to investigate inductive biases of language models using artificial languages . they use languages to create parallel corpora across languages that differ only in word order . |
| Outcome: | The proposed method shows that language models can be used to model a wide variety of languages. |
Location Attention for Extrapolation to Longer Sequences (2020.acl-main)
Copied to clipboard
| Challenge: | Neural networks are surprisingly good at interpolating, but they are often unable to extrapolate patterns beyond the seen data. |
| Approach: | They propose to use a special type of extrapolation for natural language processing to generalize to sequences that are longer than the training ones. |
| Outcome: | The proposed model is more likely to extrapolate than models with common attention mechanisms. |
Enhancing BERT for Lexical Normalization (D19-55)
Copied to clipboard
| Challenge: | Pre-trained contextual language models have improved performance of many NLP tasks. |
| Approach: | They propose to use a pre-trained language model to perform lexical normalisation without UGC resources. |
| Outcome: | The proposed model can perform lexical normalisation without the need for training sentences and 3,000 tokens. |
skweak: Weak Supervision Made Easy for NLP (2021.acl-demo)
Copied to clipboard
| Challenge: | skweak is a Python-based toolkit for NLP developers to use weak supervision . labelled data remains a scarce resource in many practical NLP scenarios . |
| Approach: | They present a Python-based toolkit that allows NLP developers to use weak supervision . skweak is designed to facilitate the use of weak supervision for NLP tasks . |
| Outcome: | skweak is a Python-based toolkit that facilitates weak supervision . the toolkit provides a simple interface to apply labels to a large corpus of text data . |
When Can Transformers Ground and Compose: Insights from Compositional Generalization Benchmarks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent benchmarks like ReaSCAN use navigation tasks grounded in a grid world to assess whether neural models exhibit compositional behaviour. |
| Approach: | They propose a transformer-based model that outperforms specialized architectures on ReaSCAN and a modified version of gSCAN to test their performance. |
| Outcome: | The proposed model outperforms specialized architectures on ReaSCAN and gSCAN on a grid world and can generalize to deeper input structures. |
Debiasing by obfuscating with 007-classifiers promotes fairness in multi-community settings (2025.coling-main)
Copied to clipboard
| Challenge: | a number of studies have focused on the mitigation of biases in text classifiers. |
| Approach: | They propose an obfuscation-based data augmentation debiasing approach to reduce bias . they add to the training data *obfuses* versions of *all* false positive instances . |
| Outcome: | The proposed approach reduces bias for almost all of the tests without sacrificing false positive rates or F1 scores for minority or majority communities. |
Evaluating Transformer Models and Human Behaviors on Chinese Character Naming (2023.tacl-1)
Copied to clipboard
| Challenge: | Neural network models have been proposed to explain the grapheme-phoneme mapping process in humans for many alphabet languages. |
| Approach: | They propose to use a dictionary-like lookup procedure to map the letter strings to their pronunciations and then use 'transformers' to capture human behavior. |
| Outcome: | The proposed models learned the correspondence of the letter strings and their pronunciation, and captured human behavior in nonce word naming tasks. |
Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing KG evaluation metrics are only aware of the exact correctness of predictions on phrase-level and ignore semantic similarities between similar predictions and targets, which inhibits the model from learning deep linguistic patterns. |
| Approach: | They propose a fine-grained evaluation metric to improve the previous KG framework . the evaluation metrics are only aware of the exact correctness of predictions on phrase-level . |
| Outcome: | The proposed method outperforms the existing frameworks among all evaluation scores. |
Social Chemistry 101: Learning to Reason about Social and Moral Norms (2020.emnlp-main)
Copied to clipboard
| Challenge: | SOCIAL CHEMISTRY is a conceptual formalism to study people’s everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language. |
| Approach: | They propose a new conceptual formalism to study people's everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language. |
| Outcome: | The proposed model can be used to model people's everyday social norms and moral judgments over a rich spectrum of real life situations. |
KGPool: Dynamic Knowledge Graph Context Selection for Relation Extraction (2021.findings-acl)
Copied to clipboard
Abhishek Nadgeri, Anson Bastos, Kuldeep Singh, Isaiah Onando Mulang’, Johannes Hoffart, Saeedeh Shekarpour, Vijay Saraswat
| Challenge: | Existing methods for relation extraction (RE) use only expanded facts from the knowledge graph . |
| Approach: | They propose a method for relation extraction from a single sentence . they use a neural network to expand the context with additional facts from the KG . |
| Outcome: | The proposed method is more accurate than state-of-the-art methods on standard datasets. |
Combining Parameter-efficient Modules for Task-level Generalisation (2023.eacl-main)
Copied to clipboard
| Challenge: | A modular design encourages neural models to disentangle and recombine different facets of knowledge to generalise more systematically to new tasks. |
| Approach: | They propose a modular neural network where a subset of latent skills is associated with a parameter-efficient model adapter. |
| Outcome: | The proposed model improves sample efficiency and few-shot generalisation in supervised learning compared to baselines. |
GloCOM: A Short Text Neural Topic Model via Global Clustering Context (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing neural topic models often overlook uncovering hidden topics from short texts due to data sparsity, poor aggregation quality, and difficulty in inferring topic proportions for individual documents. |
| Approach: | They propose a model which constructs global clustering contexts for short texts using text embeddings from pre-trained language models. |
| Outcome: | The proposed model outperforms state-of-the-art models on short texts in topic quality and document representation. |
Neural data-to-text generation: A comparison between pipeline and end-to-end architectures (D19-1)
Copied to clipboard
| Challenge: | Traditionally, data-to-text applications have been designed using a modular pipeline architecture, in which the non-linguistic input data is converted into natural language through several intermediate transformations. |
| Approach: | They propose to use Gated-Recurrent Units and Transformer to implement neural pipelines for data-to-text generation. |
| Outcome: | The proposed models generalize better to unseen inputs and have better performance than the existing pipeline architectures. |
Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that our proposed approach yields better attention mechanisms . dominant ASC models are mostly discriminative classifiers based on manual feature engineering . |
| Approach: | They propose a self-supervised approach to aspect-level sentiment classification that mines useful attention supervision information from a training corpus to refine attention mechanisms. |
| Outcome: | The proposed approach yields better attention mechanisms on multiple datasets. |
Breaking Down the Invisible Wall of Informal Fallacies in Online Discussions (2021.acl-long)
Copied to clipboard
| Challenge: | a number of people engage in unsound argumentation techniques to prove a claim on online platforms . fallacies are weak arguments that seem convincing, but their evidence does not prove or disprove the conclusion . |
| Approach: | They propose to use user comments containing fallacy mentions as noisy labels to classify fallacies . they use the pragma-dialectical theory of argumentation to study the most common fallacias on Reddit . |
| Outcome: | The proposed dataset of fallacies on reddit shows that neural models perform better in conversational context. |
Morphological Inflection with Phonological Features (2023.acl-short)
Copied to clipboard
| Challenge: | Recent advances in morphological tasks can be difficult to solve when little training data is available or when generalizing to previously unseen lemmas. |
| Approach: | They propose two methods to manipulate phonemic data to include phonological features instead of characters. |
| Outcome: | The proposed methods yield comparable results to baseline models, with minor improvements in some languages. |
Neural Collective Entity Linking (C18-1)
Copied to clipboard
| Challenge: | Entity linking aims to link entity mentions in texts to knowledge bases, but existing methods rely on local contexts to resolve entities independently. |
| Approach: | They propose a neural model for collective entity linking that integrates local contextual features and global coherence information to improve the computation efficiency. |
| Outcome: | The proposed model improves its performance on five publicly available datasets and can be used to train on Wikipedia hyperlinks to avoid overfitting and domain bias. |
Neural Data-to-Text Generation with LM-based Text Augmentation (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural data-to-text generation is a difficult task for many new applications because of a lack of training data. |
| Approach: | They propose a few-shot approach that augments the data available for training by generating new text samples based on replacing specific values by alternative ones from the same category and pairing the new text with data samples. |
| Outcome: | The proposed approach outperforms fully supervised sequence-to-sequence models with less than 10% of the training set on both datasets. |
Neural Naturalist: Generating Fine-Grained Image Comparisons (D19-1)
Copied to clipboard
| Challenge: | a dataset of 41k sentences describes fine-grained differences between photographs of birds . human observers are adept at making fine-grain comparisons, but sometimes require aid in distinguishing visually similar classes. |
| Approach: | They propose a model that generates comparative language from a dataset of 41k sentences describing fine-grained differences between photographs of birds. |
| Outcome: | The proposed model can explain differences in visual embedding space using natural language . it evaluates the results with humans who must use the descriptions to distinguish real images . |
Neural Models of Factuality (N18-1)
Copied to clipboard
| Challenge: | A central function of natural language is to convey information about the properties of events. |
| Approach: | They propose to use a FactBank, UW, and MEANTIME event factuality dataset to build two neural models that outperform previous models. |
| Outcome: | The proposed models outperform previous models on FactBank, UW, and MEANTIME datasets. |
Attention Transfer Network for Aspect-level Sentiment Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Aspect-level sentiment classification aims to detect the sentiment polarity of a given opinion target in a sentence. |
| Approach: | They propose a novel attention transfer network which can exploit attention from document-level sentiment datasets to improve the attention capability of the aspect-level classification task. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two ASC benchmark datasets. |
Consistency Regularization Training for Compositional Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing neural models have difficulty generalizing to unseen combinations of seen components. |
| Approach: | They propose to improve the capability of Transformer on compositional generalization by consistency regularization training without modifying model architectures. |
| Outcome: | The proposed model performs well on semantic parsing and machine translation benchmarks. |
The Influence of Context on Sentence Acceptability Judgements (P18-2)
Copied to clipboard
| Challenge: | a paper examining the influence of document context on acceptability judgements for English sentences is published in journal journal of linguistics. |
| Approach: | They propose to use document context to assess acceptability judgements for English sentences . they also test the accuracy of neural models that incorporate document context during training . |
| Outcome: | The proposed model improves acceptability ratings for ill-formed sentences, but reduces them for well-formed ones. |
Tree-Structured Neural Topic Model (2020.acl-main)
Copied to clipboard
| Challenge: | Existing topic models do not organize topics into coherent groups or hierarchies. |
| Approach: | They propose a tree-structured neural topic model with an infinite number of branches and a topic distribution over a forest. |
| Outcome: | The proposed model improves data scalability and competitive performance when inducing latent topics and tree structures. |
Improving Prediction Backward-Compatiblility in NLP Model Upgrade with Gated Fusion (2023.findings-eacl)
Copied to clipboard
| Challenge: | a regression error during model upgrade often outweighs the benefits of accuracy gain . a novel method that promotes backward compatibility during model upgrades is proposed . |
| Approach: | They propose a method that promotes backward compatibility via learning to mix predictions between old and new models. |
| Outcome: | The proposed method outperforms existing methods and achieves negative flip rate reductions by 73.2% on two model upgrade scenarios. |
Plan-then-Generate: Controlled Data-to-Text Generation via Planning (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on producing results that are close to the references, i.e. what to generate and in what order (the output structure) cannot be explicitly controlled by the users. |
| Approach: | They propose a Plan-then-Generate framework to improve the controllability of neural data-to-text models. |
| Outcome: | The proposed model can control both the intra-sentence and inter-sentent structure of the generated output. |
Draw Me a Flower: Processing and Grounding Abstraction in Natural Language (2022.tacl-1)
Copied to clipboard
| Challenge: | Abstraction is a core tenet of human cognition and communication. yet, interpreting and grounding abstraction expressed in natural language (NL) has not been systematically studied in NLP. |
| Approach: | They propose a 2D instruction-following game that elicits abstract instructions from 4k natural language instructions. |
| Outcome: | The proposed method elicits 4k natural language instructions rich with diverse types of abstractions and assesses neural models. |
Few-Shot Table-to-Text Generation with Prototype Memory (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Neural table-to-text generation models are data-hungry and require large amounts of training data to learn the mapping between tables and texts. |
| Approach: | They propose a framework for table-to-text generation under the few-shot scenario that uses retrieved prototypes and a prototype selector to bridge the structural gap between tables and texts. |
| Outcome: | The proposed framework significantly improves the model performance on three benchmark datasets with state-of-the-art models. |
Contextualized Query Embeddings for Conversational Search (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to conversational search use multiple inference pipelines that require long inference times . despite their effectiveness, such a pipeline often includes multiple neural models that require longer inference time. |
| Approach: | They propose to integrate conversational query reformulation directly into a dense retrieval model . they use a dataset with pseudo-relevance labels to overcome the lack of training data . |
| Outcome: | The proposed model rewrites conversational queries as dense representations in conversational search and open-domain question answering datasets. |
EDA: Enriching Emotional Dialogue Acts using an Ensemble of Neural Annotators (2020.lrec-1)
Copied to clipboard
| Challenge: | Emotion recognition helps to build natural dialogue systems. |
| Approach: | They propose to use a recurrent neural model to annotate emotion corpora with dialogue act labels and an ensemble annotator to extract the final dialogue act label. |
| Outcome: | The proposed model annotates two accessible multi-modal emotion corpora with and without context and extracts the final dialogue act label. |
Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent approaches to information retrieval (IR) and natural language processing (NLP) use contextual language models, which can improve both synonymy and polysemy problems associated with words. |
| Approach: | They propose an ultra-high dimensional representation scheme equipped with directly controllable sparsity and a bucketing method where embeddings from multiple layers of BERT are selected/merged to represent diverse linguistic aspects. |
| Outcome: | The proposed representation scheme outperforms sparse models with MS MARCO and TREC CAR, and shows that it is highly efficient for storage and search. |
Vocabulary Matters: A Simple yet Effective Approach to Paragraph-level Question Generation (2020.aacl-main)
Copied to clipboard
| Challenge: | Current neural network-based questions generation techniques take only one or two sentences as input. |
| Approach: | They propose a simple yet effective technique for question generation from paragraphs . they augment a sequence-to-sequence QG model with dynamic, paragraph-specific dictionary . |
| Outcome: | The proposed model outperforms state-of-the-art systems in question generation from paragraphs in automatic and human evaluation. |
Improving Compositional Generalization in Classification Tasks via Structure Annotations (2021.acl-short)
Copied to clipboard
| Challenge: | Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. |
| Approach: | They propose to convert a natural language sequence-to-sequence dataset into a classification dataset that requires compositional generalization. |
| Outcome: | The proposed model can generalize compositionally by providing hints on the structure of the input. |
Point Precisely: Towards Ensuring the Precision of Data in Generated Texts Using Delayed Copy Mechanism (C18-1)
Copied to clipboard
| Challenge: | Recent neural generation systems have shown significant progress on data-to-text generation tasks. |
| Approach: | They propose a two-stage approach with a delayed copy mechanism to improve the precision of data records in the generated texts. |
| Outcome: | The proposed approach improves the accuracy of the generated texts on a RotoWire dataset. |
Instance-Based Neural Dependency Parsing (2021.tacl-1)
Copied to clipboard
Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui
| Challenge: | Existing models that use instance-based inference for dependency parsing are difficult to understand for humans. |
| Approach: | They develop neural models that adopt an interpretable inference process for dependency parsing. |
| Outcome: | The proposed models achieve competitive accuracy with standard neural models and have plausibility of instance-based explanations. |
Generative Data Augmentation for Commonsense Reasoning (2020.findings-emnlp)
Copied to clipboard
Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, Doug Downey
| Challenge: | Recent advances in commonsense reasoning depend on large-scale human-authored training data. |
| Approach: | They propose a generative data augmentation technique that augments human-authored training data by using pretrained language models. |
| Outcome: | The proposed technique outperforms existing methods on commonsense reasoning benchmarks and enhances out-of-distribution generalization. |
Syntax-Aware Retrieval Augmented Code Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural code generation models with token-level retrieval capabilities are often noisy and time-consuming. |
| Approach: | They propose a token-level retrieval augmented code generation method that leverages syntax constraints for the retrieval of datastores. |
| Outcome: | The proposed method reduces the impact of retrieve noise on code generation on two datasets. |
Neural Unsupervised Reconstruction of Protolanguage Word Forms (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for reconstructing ancient word forms use expectation-maximization . past work has used this method to predict simple phonological changes . |
| Approach: | They extend expectation-maximization to predict phonological changes between ancient word forms and their cognates in modern languages. |
| Outcome: | The proposed model reduces edit distance from the target word forms compared to previous methods. |
Exploiting Document Knowledge for Aspect-level Sentiment Classification (P18-2)
Copied to clipboard
| Challenge: | Existing public aspect-level datasets for aspect-based sentiment classification are small . existing methods for aspect level sentiment classification require annotation of all opinion targets . |
| Approach: | They propose two approaches that transfer knowledge from document-level data to improve aspect-level sentiment classification. |
| Outcome: | The proposed methods improve aspect-level sentiment classification on 4 public datasets. |
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence (2021.acl-short)
Copied to clipboard
| Challenge: | Recent neural topic models extract words from documents, but they are not coherent . coherence is crucial for topic models, but many use bag-of-words document representations as input . pre-trained language models are becoming ubiquitous in natural language processing . |
| Approach: | They combine contextualized representations with neural topic models to produce more coherent topics . they say that future improvements in language models will translate into better topic models . |
| Outcome: | The proposed approach produces more meaningful and coherent topics than bag-of-words models and recent neural models. |
Enhancing Neural Models with Vulnerability via Adversarial Attack (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on adversarial attack to improve performance of NLSM tasks has not been done. |
| Approach: | They propose a general two-stage training framework to enhance neural models with Vulnerability via adversarial attack. |
| Outcome: | The proposed framework improves neural models with Vulnerability via adversarial attack on NLSM datasets. |
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks. |
| Approach: | They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity. |
| Outcome: | The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions. |
Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing studies on robustness of pretrained multilingual models are limited to the English language. |
| Approach: | They propose to use data augmentation and contrastive loss term to boost robustness of multilingual models in cross-lingual settings. |
| Outcome: | The proposed model outperforms existing models on clean and noisy data in the cross-lingual setting. |
On the Transferability of Minimal Prediction Preserving Inputs in Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work establishes the presence of short, uninterpretable input fragments that yield high confidence and accuracy in neural models. |
| Approach: | They investigate competing hypotheses for the existence of MPPIs in question answering . they discover a perplexing invariance of MPIs to random training seed, model architecture, pretraining, and training domain. |
| Outcome: | The proposed model performance is higher than comparable short queries. |
AnnoCTR: A Dataset for Detecting and Linking Entities, Tactics, and Techniques in Cyber Threat Reports (2024.lrec-main)
Copied to clipboard
Lukas Lange, Marc Müller, Ghazaleh Haratinezhad Torbati, Dragan Milchevski, Patrick Grau, Subhash Chandra Pujari, Annemarie Friedrich
| Challenge: | Abstract: Natural language processing can help with managing large amounts of unstructured information. |
| Approach: | They propose to annotate a CC-BY-SA-licensed dataset of cyber threat reports . they use named entities, temporal expressions, and cybersecurity-specific concepts . |
| Outcome: | The proposed dataset annotates reports with named entities, temporal expressions, and cybersecurity-specific concepts including implicitly mentioned techniques and tactics. |
Improving the Extraction of Supertags for Constituency Parsing with Linear Context-Free Rewriting Systems (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to parsing discontinuous constituency structures uses supertags to improve parsability . traditional approaches use grammar formalisms to model hierarchies of noncontiguous phrases . but supertags are still useful for analyzing these grammars and parsers . |
| Approach: | They propose to reformulate and parameterize extraction process for LCFRS supertags to improve parsing quality. |
| Outcome: | The proposed method improves the quality and speed of parsing with supertags over the previous method. |
TWT: Table with Written Text for Controlled Data-to-Text Generation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods output hallucinated text that is not faithful on TWT. |
| Approach: | They propose to generate text conditioned on the structured data and a prefix by leveraging pre-trained neural models. |
| Outcome: | The proposed approach outperforms state-of-the-art methods under automatic and human evaluation metrics. |
Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models (N18-2)
Copied to clipboard
| Challenge: | Current diagnoses often involve lengthy medical evaluations. |
| Approach: | They apply neural models based on CNNs, LSTM-RNNs, and their combination to classify AD and control language samples. |
| Outcome: | The proposed model achieves independent benchmark accuracy for the AD classification task. |
Evaluating Historical Text Normalization Systems: How Well Do They Generalize? (N18-2)
Copied to clipboard
| Challenge: | Historical text normalization systems aim to convert historical wordforms to their modern equivalents . many of these systems have been developed and tested on a single language . |
| Approach: | They propose to use a nave baseline system to evaluate historical text normalization systems . they show that the models generalize well to unseen words in tests on five languages . |
| Outcome: | The proposed models generalize well to unseen words on five languages, but provide no clear benefit over the nave baseline. |
Split and Rephrase: Better Evaluation and Stronger Baselines (P18-2)
Copied to clipboard
| Challenge: | a dataset mapping a complex sentence to a sequence of sentences conveying the same meaning is challenging in NLP. |
| Approach: | They propose a neural split and a copy-mechanism to break a complex sentence into several shorter sentences that convey the same meaning. |
| Outcome: | The proposed model outperforms the baseline model by 8.68 BLEU and further improves on the task. |
AdaPT: A Set of Guidelines for Hyperbolic Multimodal Multilingual NLP (2024.findings-naacl)
Copied to clipboard
| Challenge: | Euclidean space is used for training neural models and performing arithmetic operations, but many data types have complex geometries and cannot be captured in the Euclidesan space. |
| Approach: | They propose a set of guidelines for initialization, parametrization, and training of neural networks that can be generalized over existing neural network training methodologies. |
| Outcome: | The proposed framework outperforms Euclidean methods on three tasks over 12 languages and modalities on a variety of domains. |
Searching for Search Errors in Neural Morphological Inflection (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural sequence-to-sequence models are the predominant choice for language generation tasks. |
| Approach: | They find that on word-level tasks, the empty string is often the global optimum . they suggest that the poor calibration of many neural models may stem from characteristics of a specific subset of tasks rather than general ill-suitedness of such models for language generation. |
| Outcome: | The results suggest that the poor calibration of many neural models may stem from characteristics of a specific subset of tasks rather than general ill-suitedness of such models for language generation. |
DoLFIn: Distributions over Latent Features for Interpretability (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to interpret neural networks face a trade-off between a model's usefulness and its complexity. |
| Approach: | They propose a novel approach to achieve interpretability that avoids this trade-off by using probability as the central quantity instead of a fixed quantity. |
| Outcome: | The proposed approach outperforms the classical CNN and BiLSTM classifiers on the SST2 and AG-news datasets. |
Conditional Neural Generation using Sub-Aspect Functions for Extractive News Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text summarization have overcome position bias in news articles . however, there are long-standing, unresolved challenges in extractive summarizing . |
| Approach: | They propose a neural framework that can flexibly control summary generation by introducing a set of sub-aspect functions. |
| Outcome: | The proposed framework can flexibly control summary generation by introducing sub-aspect functions . extracted summaries with minimal position bias are comparable with standard models . |
The (Undesired) Attenuation of Human Biases by Multilinguality (2022.emnlp-main)
Copied to clipboard
| Challenge: | odor pleasantness perception is universal, but cultural biases are not always present in embedding models . et al., 2018: a new study shows that cultural bias is not always the case in embedded models based on human texts . |
| Approach: | They propose multilingual cultural aware tests to quantify biases in embedding models . they find that biased models are more likely to be multilingual than monolingual ones . |
| Outcome: | The results show that human preferences are not always universal . they also show that multilinguality reverses biases, despite differences in training corpus . |
Original Content Is All You Need! an Empirical Study on Leveraging Answer Summary for WikiHowQA Answer Selection Task (2022.coling-1)
Copied to clipboard
Liang Wen, Juan Li, Houfeng Wang, Yingwei Luo, Xiaolin Wang, Xiaodong Zhang, Zhicong Cheng, Dawei Yin
| Challenge: | Existing answer selection approaches for community question answering lack additional answer summaries due to redundancy and lengthiness issues of crowdsourced answers. |
| Approach: | They constructed a dataset which contains a corresponding reference summary for each original lengthy answer. |
| Outcome: | The proposed model improves the performance of a question and candidate answer on a WikiHowQA dataset. |
Semantic Linking in Convolutional Neural Networks for Answer Sentence Selection (D18-1)
Copied to clipboard
| Challenge: | Recent NLP approaches that model relations between text use complex architectures and attention. |
| Approach: | They propose to use labelled data to model semantic relations between two pieces of text . they use word representations to encode matching features directly in the word representation . |
| Outcome: | The proposed approach beats tree kernel models and neural models with similar input encodings while keeping the model simple and fast to train. |
Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to encode natural languages without orders are lacking. |
| Approach: | They conduct a comprehensive analysis of the ability of neural models to organize sentences from a bag of words under three typical scenarios. |
| Outcome: | The proposed models can reorder or reconstruct sentences from a bag of words under three typical scenarios. |
The Role of Semantic Parsing in Understanding Procedural Text (2023.findings-eacl)
Copied to clipboard
| Challenge: | Inferring actions and their impact on entities involved in a procedural text can be challenging in various aspects. |
| Approach: | They propose a symbolic parser and semantic role labeling as two sources of semantic parsing knowledge. |
| Outcome: | The proposed framework integrates semantic parsing knowledge into state-of-the-art neural models and shows that it improves procedural understanding. |
White-to-Black: Efficient Distillation of Black-Box Adversarial Attacks (N19-1)
Copied to clipboard
| Challenge: | Recent work in natural language processing generates adversarial examples using white-box access . a neural network can learn to emulate the behavior of a white- box attack and generalize well to new examples. |
| Approach: | They propose an adversarial training approach that assumes white-box access to an attacker's model and optimizes the input directly against it. |
| Outcome: | The proposed approach reduces example generation time by 19x-39x and exposes the Google Perspective API vulnerability. |
Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on sentence prediction tasks uses shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss. |
| Approach: | They propose to use a pre-trained language model to learn text representations first and then to constrain the scores with regression loss or ranking loss. |
| Outcome: | The proposed model outperforms state-of-the-art models on the Automated Student Assessment Prize dataset. |
All You May Need for VQA are Image Captions (2022.naacl-main)
Copied to clipboard
| Challenge: | Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. |
| Approach: | They propose a method that automatically derives VQA examples at volume by leveraging existing image-caption annotations combined with neural models for textual question generation. |
| Outcome: | The proposed method improves state-of-the-art zero-shot accuracy by double digits and achieves robustness that lacks in the same model trained on human-annotated VQA data. |
Tracking State Changes in Procedural Text: a Challenge Dataset and Models for Process Paragraph Comprehension (N18-1)
Copied to clipboard
| Challenge: | Using synthetic data, existing models struggle with questions that require inference. |
| Approach: | They propose a dataset and two new neural models that exploit alternative mechanisms for state prediction. |
| Outcome: | The proposed dataset improves accuracy by 19% over previous models. |
Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks (P19-1)
Copied to clipboard
| Challenge: | Existing models for natural language processing are heavily parameterized and memory inefficient. |
| Approach: | They propose a series of lightweight and memory efficient neural architectures for NLP tasks . they propose quaternion algebra and hypercomplex spaces for computation . |
| Outcome: | The proposed models enable expressive inter-component interactions and significantly reduce parameter size without loss of performance. |
On the Robustness of Self-Attentive Models (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that self-attentive neural models are more robust against adversarial perturbations compared to recurrent neural networks. |
| Approach: | They propose an adversarial attack algorithm that generates more natural adversarials . they propose to use the attention mechanism to learn a context-dependent representation . |
| Outcome: | The proposed attack algorithm generates more natural adversarial examples that could mislead models but not humans. |
Recognizing Humour using Word Associations and Humour Anchor Extraction (C18-1)
Copied to clipboard
| Challenge: | Using humour anchors to improve the performance of humor recognition and interpretation is difficult for computers. |
| Approach: | They propose to use word associations to improve humour recognition models by using humor anchors to improve the performance of semantic features. |
| Outcome: | The proposed models improve the performance of humour recognition and interpretation tasks. |
Complaint Identification in Social Media with Transformer Networks (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work on identifying complaints in social media has focused on feature-based and task-specific neural network models. |
| Approach: | They evaluate a battery of neural models underpinned by transformer networks and combine them with linguistic information to predict complaints. |
| Outcome: | The proposed models outperform state-of-the-art methods on a publicly available dataset achieving a macro F1 up to 87. |
Inflecting When There’s No Majority: Limitations of Encoder-Decoder Neural Networks as Cognitive Models for German Plurals (2020.acl-main)
Copied to clipboard
| Challenge: | Encoder-decoder models can be used to generalize to inflectional morphology and generalize new words, but they fail on tasks like German number inflection, where infrequent suffixes like /-s/ can still be productively generalized. |
| Approach: | They propose to use a dataset to collect data from German speakers to examine whether ED models can generalize the most frequently produced plural class. |
| Outcome: | The proposed model does not show human-like variability or ‘regular’ extension of other plural markers. |
Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity recognition models use a conditional random field as the final layer . current work eschews prior knowledge of how the span encoding scheme works . |
| Approach: | They propose to constrain the output to suppress illegal transitions to train a tagger with a cross-entropy loss twice as fast as a CRF. |
| Outcome: | The proposed model trains twice as fast as a CRF with statistically insignificant differences in F1 . the proposed model is open source and can be used in PyTorch and TensorFlow. |
Neural Duplicate Question Detection without Labeled Training Data (D19-1)
Copied to clipboard
| Challenge: | Recent studies have used alternative methods to train neural models to duplicate question detection in community Question Answering forums. |
| Approach: | They propose two new methods for supervised question detection in community Question Answering forums . they propose weak supervision using title and body of question and automatic generation of duplicate questions . |
| Outcome: | The proposed methods can achieve better performance even without labeled data. |
I Beg to Differ: A study of constructive disagreement in online conversations (2021.eacl-main)
Copied to clipboard
| Challenge: | Disagreements are pervasive in human communication. |
| Approach: | They construct a corpus of Wikipedia Talk page conversations that contain content disputes and define the task of predicting whether disagreements will be escalated to mediation by a moderator. |
| Outcome: | The proposed model outperforms feature-based models in predicting whether disagreements will escalate to mediation by a moderator. |
Fixed That for You: Generating Contrastive Claims with Semantic Edits (N19-1)
Copied to clipboard
| Challenge: | Understanding contrastive opinions is a key component of argument generation. |
| Approach: | They create a corpus of Reddit comment pairs and train neural models to edit the original claim and produce a new claim with a different view. |
| Outcome: | The proposed model improves on a sequence-to-sequence baseline and compared to a human evaluation for fluency, coherence, and contrast. |
Picking Apart Story Salads (D18-1)
Copied to clipboard
| Challenge: | Story salads are mixtures of multiple documents that can be generated at scale . they exhibit challenging inference problems, and require global context and coherence . |
| Approach: | They propose to generate salads that exhibit challenging inference problems by exploiting the Wikipedia hierarchy . they propose a task where the objective is to group sentences from the same narratives . |
| Outcome: | The proposed task is based on a novel, challenging clustering task using Wikipedia . it is difficult to identify relevant information and assemble it into coherent narratives . |
PAUQ: Text-to-SQL in Russian (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Semantic parsing is an important task that allows to democratize human-computer interaction. |
| Approach: | They construct and complement a Russian text-to-SQL dataset by integrating a spider query with a RAT-SqL and BRIDGE database. |
| Outcome: | The proposed datasets show that they perform well with monolingual training and improved accuracy in multilingual scenarios. |
Probing Linguistic Systematicity (2020.acl-main)
Copied to clipboard
| Challenge: | Existing evidence that deep natural language understanding models do not learn systematically is lacking. |
| Approach: | They examine whether deep natural language understanding models exhibit systematicity . they find that network architectures can generalize non-systematically . |
| Outcome: | The proposed model generalizes non-systematically, but is unsatisfactory, the authors argue . they show that the current state-of-the-art models do not generalize systematically . |
Analytical Reasoning of Text (2022.findings-naacl)
Copied to clipboard
Wanjun Zhong, Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Yining Chen, Jiahai Wang, Jian Yin, Ming Zhou, Nan Duan
| Challenge: | Existing models with implicit reasoning ability struggle to solve analytical reasoning of text. |
| Approach: | They propose an approach to analyze text and use it to perform reasoning over it. |
| Outcome: | The proposed approach outperforms pre-trained models on an analysis of the Law School Admission Test dataset. |
Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent advances in the area of long document matching have primarily focused on using transformer-based models for long document encoding and matching. |
| Approach: | They propose to use simple neural models and simple embeddings to improve document matching by taking significantly less training time, energy, and memory. |
| Outcome: | The proposed models outperform the more complex BERT-based models while taking significantly less training time, energy, and memory. |
BC-Prover: Backward Chaining Prover for Formal Theorem Proving (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for interactive theorem proving in formal logic lack robustness and robustness. |
| Approach: | They propose a backward chaining framework guided by pseudo steps for proofstep generation that prioritizes pseudo steps. |
| Outcome: | The proposed framework improves on the miniF2F benchmark. |
Combining Distant and Direct Supervision for Neural Relation Extraction (N19-1)
Copied to clipboard
| Challenge: | Existing methods to train relation extraction with distant supervision use noisy labels and implicitly assumes that all the KB facts are mentioned in the text. |
| Approach: | They propose to combine distant supervision data with additional directly-supervised data to train relation extraction models by using sigmoidal attention weights with max pooling. |
| Outcome: | The proposed method achieves state-of-the-art on the widely used FB-NYT dataset. |
Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to generate source code summaries are coarse-grained and noise-filled . however, they do not capture contextual code semantics and are often outdated in continuous software iteration. |
| Approach: | They propose a fine-grained Token-level retrieval-augmented mechanism on the decoder side to enhance performance of neural models. |
| Outcome: | The proposed method produces more low-frequency tokens and is interpretable. |
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation (2022.findings-acl)
Copied to clipboard
Tianxiang Sun, Xiangyang Liu, Wei Zhu, Zhichao Geng, Lingling Wu, Yilong He, Yuan Ni, Guotong Xie, Xuanjing Huang, Xipeng Qiu
| Challenge: | Existing methods to measure instance difficulty use generalization and threshold-tuning . a new approach to learn to exit is based on hash functions to assign tokens to a fixed exiting layer. |
| Approach: | They propose a Hash-based Early Exiting approach that replaces learn-to-exit modules with hash functions to assign each token to a fixed exiting layer. |
| Outcome: | The proposed approach improves on learning to exit and predicting instance difficulty. |
Neural text normalization leveraging similarities of strings and sounds (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods that ignore the similarities of word strings and sounds do not account for these features. |
| Approach: | They propose a neural model that considers the similarities of both word strings and sounds, and a model that takes only the similarity of word strings or of sounds as a baseline. |
| Outcome: | The proposed models outperformed a baseline model and achieved state-of-the-art results on WNUT-2015. |
When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP (2024.acl-long)
Copied to clipboard
| Challenge: | despite its crucial role in research experiments, code correctness is often presumed on the perceived quality of results. |
| Approach: | They propose to promote code-quality checklists to promote coding best practices . they propose to fix bugs in conformer implementations to mitigate this risk . |
| Outcome: | The proposed checklists aim to promote coding best practices and improve software quality within the NLP community. |
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios (2021.naacl-main)
Copied to clipboard
| Challenge: | a growing body of work is focused on improving performance in low-resource settings . a goal of this study is to explain how these methods differ in their requirements . |
| Approach: | They propose to analyze data-lean scenarios across different dimensions of data availability to understand which approaches are effective in a specific low-resource setting. |
| Outcome: | The proposed methods enable learning when training data is sparse. |
Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation (2020.emnlp-main)
Copied to clipboard
Tahmid Hasan, Abhik Bhattacharjee, Kazi Samin, Masum Hasan, Madhusudan Basak, M. Sohel Rahman, Rifat Shahriyar
| Challenge: | despite being the seventh most widely spoken language, Bengali has received little attention in machine translation due to being low in resources. |
| Approach: | They propose a customized sentence segmenter for Bengali and two new methods for parallel corpus creation on low-resource setups. |
| Outcome: | The proposed method improves Bengali-English parallel corpus by 9 BLEU over previous approaches . the results will pave the way for future research on Bengali and other low-resource languages . |
Chat-Ghosting: Methods for Auto-Completion in Dialog Systems (2026.eacl-long)
Copied to clipboard
| Challenge: | Ghosting is a type-ahead completion task that predicts a user's intended input for inline query auto-completion (QAC). |
| Approach: | They propose to use ghosting to predict a user's intended input for inline query auto-completion by suggesting completions to incomplete queries. |
| Outcome: | The proposed method outperforms deep learning and deep learning methods with and without dialog context for ghosting. |
Ruddit: Norms of Offensiveness for English Reddit Comments (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to detect offensive language have been limited by categorical labels . however, there are several challenges in the detection of such content . |
| Approach: | They analyze Reddit comments with fine-grained, real-valued offensiveness scores . they evaluate the ability of widely-used neural models to predict offensiveness . |
| Outcome: | The proposed method produces highly reliable offensiveness scores and can predict scores on reddit comments. |
WikiNEuRal: Combined Neural and Knowledge-based Silver Data Creation for Multilingual NER (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a key intermediate task in NLP. |
| Approach: | They propose a method which uses knowledge-based approaches and neural models to produce high-quality training corpora for NER. |
| Outcome: | The proposed method improves on standard benchmarks and yields significant improvements up to 6 span-based F1-score points over previous state-of-the-art systems for data creation. |
Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models (2020.lrec-1)
Copied to clipboard
| Challenge: | a recent study finds brittleness in explanations obtained through attention mechanisms . a philosophy of science theory allows robust yet non-causal reasoning in explanation . |
| Approach: | They propose to use philosophy of science to examine the state-of-the-art in explanation for NLP models . they argue that it is impossible to explain attention-based learning by attention mechanisms . |
| Outcome: | The proposed model selection criteria are based on philosophy of science theories . the proposed model is based upon a model that is more explainable than a classical model . |
Neural Generation of Dialogue Response Timings (2020.acl-main)
Copied to clipboard
| Challenge: | Using neural models, the timings of spoken response offsets in human dialogue can vary based on contextual elements of the dialogue. |
| Approach: | They propose neural models that simulate the distributions of response offsets taking into account the response turn as well as the preceding turn. |
| Outcome: | The proposed models can generate distributions of response offsets based on the response turn and preceding turn based upon human listening tests and offline experiments. |
Misinfo Reaction Frames: Reasoning about Readers’ Reactions to News Headlines (2022.acl-long)
Copied to clipboard
Saadia Gabriel, Skyler Hallinan, Maarten Sap, Pemi Nguyen, Franziska Roesner, Eunsol Choi, Yejin Choi
| Challenge: | Empirical results confirm that it is indeed possible for neural models to predict the prominent patterns of readers’ reactions to previously unseen news headlines. |
| Approach: | They propose a pragmatic formalism for modeling how readers might react to a news headline . they propose 'misinfo' frames, which can be used to model reader perceptions of news reliability . |
| Outcome: | The proposed model can predict readers' reactions to previously unseen headlines. |
Controlled Neural Sentence-Level Reframing of News Articles (2021.findings-emnlp)
Copied to clipboard
| Challenge: | a news article is framed from a specific perspective, but reframing can be difficult . a framed article can be used to communicate with opposing camps of audiences . |
| Approach: | They propose to reframe news articles using a media frame corpus to achieve this . they propose three strategies to train neural models for reframing . |
| Outcome: | The proposed techniques maintain coherence of sentences and reframe them correctly . the proposed techniques are effective but have tradeoffs . |
Neural Text Generation from Rich Semantic Representations (N19-1)
Copied to clipboard
| Challenge: | 2 is a neural model that maps a linearization of Dependency MRS to text . 1 is based on a BLEU score of 66.11 when trained on gold data . |
| Approach: | They propose to use Minimal Recursion Semantics to generate high-quality text from structured representations. |
| Outcome: | The proposed model achieves a BLEU score of 77.17 on the full test set and 83.37 on the subset of test data most closely matching the silver data domain. |
UMSE: Unified Multi-scenario Summarization Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Summarization quality evaluation is a non-trivial task in text summarization. |
| Approach: | They propose a unified multi-scenario summarization evaluation model that shares cross-sceenario knowledge and uses a self-supervised training paradigm to optimize the model without extra human labeling. |
| Outcome: | The proposed model can achieve comparable performance with existing methods for three evaluation scenarios. |
Large-Scale Hate Speech Detection with Cross-Domain Transfer (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for hate speech detection are limited due to the labor cost. |
| Approach: | They construct large-scale tweet datasets for hate speech detection in English and a low-resource language, Turkish, consisting of human-labeled 100k tweets per each. |
| Outcome: | The proposed datasets outperform conventional bag-of-words and neural models by at least 5% in English and 10% in Turkish for large-scale hate speech detection. |
Multimodal Semi-supervised Learning for Disaster Tweet Classification (2022.coling-1)
Copied to clipboard
| Challenge: | During natural disasters, people use social media platforms to post information about casualties and damage . annotating data can be burdensome, subjective and expensive . et al., 2018b; sohn e.t., 2020) proposed semi-supervised multimodal approach to improve performance on multimodal tasks. |
| Approach: | They propose a semi-supervised approach to annotate unlabeled data from Twitter . they extend FixMatch algorithm to a multimodal setting to account for subjective data . |
| Outcome: | The proposed approach improves on multimodal disaster tweet classification tasks. |
Teacher and Student Models of Offensive Language in Social Media (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to identify offensive language online use large pre-trained transformer models. however, the inference time, disk, and memory requirements of these models are prohibitively large. |
| Approach: | They propose to transfer knowledge from large transformer models to much smaller neural models to make predictions at the token- and post-level. |
| Outcome: | The proposed model performs 100 times better than transformer models but with 100 times less parameters and much less memory usage. |
Pre- and In-Parsing Models for Neural Empty Category Detection (P18-1)
Copied to clipboard
| Challenge: | Existing studies on empty category detection have shown positive effects on syntactic parsing . empty categories are used to indicate long-distance dependencies, discontinuous constituents, and certain dropped elements. |
| Approach: | They propose to use ECD to detect empty categories without syntactic analysis. |
| Outcome: | The proposed models outperform the prior state-of-the-art by significant margins. |
Evaluating Theory of Mind in Question Answering (D18-1)
Copied to clipboard
| Challenge: | a dataset is proposed for question answering models with respect to their capacity to reason about beliefs. |
| Approach: | They propose a dataset for evaluating question answering models with respect to their capacity to reason about beliefs. |
| Outcome: | The proposed dataset is inspired by theory-of-mind experiments that examine whether children are able to reason about beliefs of others. |
Narrate Dialogues for Better Summarization (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on dialogue summarization models focuses on generating concise summaries for multi-party dialogues. |
| Approach: | They propose several ways to convert dialogue into a third-person narrative style . they propose to use narration as a valuable annotation for LLMs . |
| Outcome: | Empirical results show that the proposed approach achieves higher scores on ROUGE and a factual correctness metric. |
Semantics as a Foreign Language (D18-1)
Copied to clipboard
| Challenge: | (2017): Syntactic grammars capture propositions, but graph-based representations aim to capture a wider notion of propositions. |
| Approach: | They propose a neural sequence-to-sequence framework which can recover syntactic linearizations by a sequence-based approach. |
| Outcome: | The proposed framework performs almost on-par with previous state-of-the-art approaches while requiring less parallel training annotations. |
Challenges with Sign Language Datasets for Sign Language Recognition and Translation (2022.lrec-1)
Copied to clipboard
Mirella De Sisto, Vincent Vandeghinste, Santiago Egea Gómez, Mathieu De Coster, Dimitar Shterionov, Horacio Saggion
| Challenge: | Sign Languages are the primary means of communication for at least half a million people in Europe . however, the development of SL recognition and translation tools is slowed down by resource scarcity and data formats are not suitable for machine learning. |
| Approach: | They propose a framework to unify available resources and facilitate SL research for different languages. |
| Outcome: | The proposed framework is based on a set of ELAN files and returns textual and visual data ready to train SL recognition and translation models. |
Strong and Simple Baselines for Multimodal Utterance Embeddings (N19-1)
Copied to clipboard
| Challenge: | Human language is a rich multimodal signal consisting of spoken words, facial expressions, body gestures, and vocal intonations. |
| Approach: | They propose two simple but strong baselines to learn embeddings of multimodal utterances by factorizing the utterant into unimodal factors. |
| Outcome: | The proposed models show that they can be derived in closed form while maintaining simplicity and efficiency during learning and inference. |
Event Transition Planning for Open-ended Text Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Open-ended text generation tasks require models to generate coherent continuation given limited preceding context. |
| Approach: | They propose a novel two-stage method which explicitly arranges ensuing events in open-ended text generation tasks. |
| Outcome: | The proposed method improves coherence and diversity of open-ended text generation tasks. |
What is Stigma Attributed to? A Theory-Grounded, Expert-Annotated Interview Corpus for Demystifying Mental-Health Stigma (2025.acl-long)
Copied to clipboard
| Challenge: | Existing resources for training neural models to finely classify mental-health stigma are limited, relying primarily on social media or synthetic data without theoretical underpinnings. |
| Approach: | They propose to use an expert-annotated corpus of human-chatbot interviews to finely classify mental-health stigma. |
| Outcome: | The proposed corpus can facilitate research on computationally detecting, neutralizing, and counteracting mental-health stigma. |
Bridging the Training-Inference Gap for Dense Phrase Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for building dense retrievers are often misaligned and do not reflect retrieval scenario at inference time. |
| Approach: | They propose a way to validate dense retrievers using a small subset of the entire corpus. |
| Outcome: | The proposed model improves top-1 phrase retrieval accuracy by 2 3 points and top-20 passage retrieval by 2 4 points for open-domain question answering. |
Exploring Author Context for Detecting Intended vs Perceived Sarcasm (P19-1)
Copied to clipboard
| Challenge: | Existing studies on textual sarcasm detection use manual labelling and tag-based distant supervision to detect sarcasm. |
| Approach: | They define author context as the embedded representation of their historical tweets and suggest neural models that extract these representations. |
| Outcome: | The proposed models achieve state-of-the-art on two datasets labelled manually and via tag-based distant supervision indicating a difference between intended and perceived sarcasm . |
MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to infer logical relations with annotated training data suffer from over-fitting and poor generalization problems due to the dataset sparsity. |
| Approach: | They propose a MEta-path guided contrastive learning method for logical ReasonIng of text that performs self-supervised pre-training on abundant unlabeled text data. |
| Outcome: | The proposed method outperforms the baselines on two logical reasoning benchmarks with significant improvements. |
Chart-to-Text: A Large-Scale Benchmark for Chart Summarization (2022.acl-long)
Copied to clipboard
Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, Shafiq Joty
| Challenge: | Inferring key insights from charts can be challenging and time-consuming. |
| Approach: | They propose a task where the goal is to explain a chart and summarize key takeaways from it in natural language. |
| Outcome: | The proposed model produces fluent summaries but suffers from hallucinations and factual errors . the proposed model is compared with other models and can be used to generate BLEU scores . |
Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing research has focused on training open-domain dialogue models using unpaired data. |
| Approach: | They propose a data-level distillation method for training open-domain dialogue models by utilizing unpaired data. |
| Outcome: | The proposed method produces high-quality dialogue pairs with diverse contents, and can improve competitive baselines. |
Learning to Model Editing Processes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence generation models produce outputs in one pass, usually left-to-right . current models model only a single edit step, and do not fully model editing . |
| Approach: | They propose to model editing processes, modeling the whole process of iteratively generating sequences. |
| Outcome: | The proposed model improves performance on a variety of axes compared to previous models . iterative refinement and editing are central parts of human creative workflow . |
Is Neural Topic Modelling Better than Clustering? An Empirical Study on Clustering with Contextual Embeddings for Topics (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent work incorporates pre-trained word embeddings into Neural Topic Models (NTMs), generating highly coherent topics. |
| Approach: | They conduct thorough experiments to investigate whether embeddings directly with an appropriate word selection method can generate more coherent and diverse topics than NTMs. |
| Outcome: | The proposed model generates more coherent and diverse topics than traditional NTMs, achieving higher efficiency and simplicity. |
Non-compositional Expression Generation Based on Curriculum Learning and Continual Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Non-compositional expressions are a classic ‘pain in the neck’ for NLP systems because of their non-composibility and limited data resources. |
| Approach: | They propose a dynamic curriculum learning framework which learns training examples from easy ones to harder ones but suffers from the forgetting problem. |
| Outcome: | The proposed framework improves on idiomatic expression generation and metaphor generation. |
Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks (2021.naacl-main)
Copied to clipboard
| Challenge: | Masked language models have become the de facto standard when processing text . however, these models are evaluated in a monolingual setting only . |
| Approach: | They propose a language-independent entity prediction task as an intermediate training procedure to ground word representations on entity semantics and bridge the gap between different languages. |
| Outcome: | The proposed approach bridges the gap between word representations and knowledge graphs by using a shared vocabulary of entities. |
Local Structure Matters Most: Perturbation Study in NLU (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent research shows that neural models are insensitive to word-order perturbations, but other studies suggest that models learn some abstract notion of syntax. |
| Approach: | They develop order-altering perturbations on the order of words, subwords, and characters to analyze their effect on neural models’ performance on language understanding tasks. |
| Outcome: | The proposed models are insensitive to word-order perturbations while the local ordering remains relatively unperturbed. |
Dynamically Disentangling Social Bias from Task-Oriented Representations with Adversarial Attack (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods to learn representations from text often reflect social biases . previous methods rely on pre-specified direction or suffer from unstable training . |
| Approach: | They propose an adversarial disentangled debiasing model to decouple social bias attributes from intermediate representations trained on the main task. |
| Outcome: | The proposed model decouples social bias attributes from intermediate representations trained on the main task. |
Probing Factually Grounded Content Transfer with Factual Ablation (2022.findings-acl)
Copied to clipboard
| Challenge: | Despite recent success, large neural models often generate factually incorrect text . lack of a standard evaluation for factuality complicates factual grounded generation . |
| Approach: | They propose a method to measure factual consistency by presenting two evaluation sets . large pretrained models have shown impressive effectiveness at longstanding tasks . |
| Outcome: | The proposed method improves over strong baselines by presenting two evaluation sets. |
Mapping probability word problems to executable representations (2021.emnlp-main)
Copied to clipboard
Simon Suster, Pieter Fivez, Pietro Totis, Angelika Kimmig, Jesse Davis, Luc de Raedt, Walter Daelemans
| Challenge: | a recent paper addresses the problem of solving math word problems automatically . a number of approaches have been proposed for solving word problems . |
| Approach: | They employ a sequence-to-sequence model to generate intermediate representations for word problems . they then use a probabilistic programming system to provide the answer . their best performing model incorporates general-domain contextualised word representations . |
| Outcome: | The proposed model is the best performing on a declarative language and a probabilistic programming system. |
Enhancing Neural Data-To-Text Generation Models with External Background Knowledge (D19-1)
Copied to clipboard
| Challenge: | Recent neural models for data-to-text generation rely on parallel pairs of data and text to learn writing knowledge. |
| Approach: | They propose to enhance neural models with external knowledge to improve fidelity of generated text. |
| Outcome: | The proposed model improves on Wikipedia infobox-to-text datasets on 21 datasets. |
TAN-NTM: Topic Attention Networks for Neural Topic Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Topic models have been widely used to learn text representations and gain insight into document corpora. |
| Approach: | They propose a framework which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. |
| Outcome: | The proposed model improves on two downstream tasks: document classification and topic guided keyphrase generation. |
A Neural Pairwise Ranking Model for Readability Assessment (2022.findings-acl)
Copied to clipboard
| Challenge: | Automatic Readability Assessment (ARA) is traditionally treated as a classification problem in NLP research. |
| Approach: | They propose a neural ranking approach to automatic readability assessment (ARA) they propose 'neural' ranking methods that can be used to rank texts by reading level . |
| Outcome: | The proposed approach performs well in monolingual single/cross corpus testing scenarios and achieves a zero-shot cross-lingual ranking accuracy of over 80% for both French and Spanish when trained on English data. |
“We will Reduce Taxes” - Identifying Election Pledges with Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | a political party's manifestos are published before any election, but do they follow through? a new study uses neural models to distinguish between actual pledges and general statements . |
| Approach: | They use election manifestos of Swedish and Indian political parties to learn neural models that distinguish actual pledges from generic positions. |
| Outcome: | The proposed model can predict election year and manifesto's party, while context introduces noise. |
Zero-Shot Cross-Lingual Abstractive Sentence Summarization through Teaching Generation and Attention (P19-1)
Copied to clipboard
| Challenge: | Abstractive Sentence Summarization (ASSUM) is a monolingual task that focuses on grasping the core idea of the source sentence and presenting it as the summary. |
| Approach: | They propose to use monolingual ASSUM to train a cross-lingual ASL system . they propose to train the system on summary word generation and attention . |
| Outcome: | Experiments show that the proposed method improves on the monolingual ASSUM task. |
Revisit Overconfidence for OOD Detection: Reassigned Contrastive Learning with Adaptive Class-dependent Threshold (2022.naacl-main)
Copied to clipboard
Yanan Wu, Keqing He, Yuanmeng Yan, QiXiang Gao, Zhiyuan Zeng, Fujia Zheng, Lulu Zhao, Huixing Jiang, Wei Wu, Weiran Xu
| Challenge: | Existing methods for OOD detection are based on labeled in-domain data . detecting out-of-domain (OOD) or unknown intents is challenging . |
| Approach: | They propose a novel reassigned contrastive learning method to discriminate IND intents for over-confident OOD and an adaptive class-dependent local threshold mechanism to separate similar IND and OOD intents. |
| Outcome: | The proposed method is effective for both aspects of overconfidence issues. |
An LLM Feature-based Framework for Dialogue Constructiveness Assessment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on dialogue constructiveness assessment focus on analysing conversational factors that influence individuals to take specific actions, win debates, change their perspectives or broaden their open-mindedness. |
| Approach: | They propose an LLM feature-based framework for dialogue constructiveness assessment that combines the strengths of feature- and neural approaches while mitigating their downsides. |
| Outcome: | The proposed framework outperforms standard feature-based models and neural models on three dialogue constructiveness datasets. |
Point to the Expression: Solving Algebraic Word Problems using the Expression-Pointer Transformer Model (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that generate solution equations using ‘Op (operator/operand) tokens suffered expression fragmentation and operand-context separation. |
| Approach: | They propose a pure neural model, Expression-Pointer Transformer, which uses (1) ‘Expression’ token and (2) operand-context pointers when generating solution equations. |
| Outcome: | The proposed model achieves comparable performance accuracy to state-of-the-art models and achieves better performance than existing models by at most 40%. |
An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Neural language models are often trained on positive examples, but recent studies suggest they are not robust enough to handle complex syntactic constructions. |
| Approach: | They propose to use negative examples to boost models' robustness on English sentences with a negligible loss of perplexity. |
| Outcome: | The proposed model is robust to negative examples in English with negligible loss of perplexity . |
Transforming Term Extraction: Transformer-Based Approaches to Multilingual Term Extraction Across Domains (2021.findings-acl)
Copied to clipboard
| Challenge: | Automated Term Extraction (ATE) is a challenging task, with few exceptions. |
| Approach: | They propose to use a transformer-based term extraction model to extract terms from sentences . they also propose to employ a language model for token classification and a sequence model to reduce sentences to terms . |
| Outcome: | The proposed models outperform baselines on the ATE challenge TermEval 2020 dataset in English, French, and Dutch. |
Analyzing Online Political Advertisements (2021.findings-acl)
Copied to clipboard
| Challenge: | Online political advertising is an integral part of modern digital election campaigning. |
| Approach: | They propose to use textual and visual information from pre-trained neural models to infer the political ideology of an ad sponsor and identify whether the sponsor is an official political party or a third-party organization. |
| Outcome: | The proposed approach outperforms state-of-the-art methods for generic commercial ad classification and linguistic analysis to study the characteristics of political ads discourse. |
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change (2020.acl-main)
Copied to clipboard
| Challenge: | Compared to previous studies, the performance of neural models is likely to be affected by the choice of hyper-parameters. |
| Approach: | They propose to automatically and dynamically determine batch sizes by accumulating gradients of mini-batches and performing an optimization step at just the time when the direction of gradients starts to fluctuate. |
| Outcome: | The proposed approach improves the Transformer model with a fixed 25k batch size by +0.73 and +0.82 BLEU respectively. |
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models (2021.naacl-main)
Copied to clipboard
| Challenge: | a common approach to coherence evaluation is shuffling the sentence order of a text, creating incoherent text samples that need to be discriminated from the original. |
| Approach: | They propose an extendable set of test suites addressing different aspects of discourse and dialogue coherence. |
| Outcome: | The proposed evaluation paradigm is suited to evaluate linguistic qualities that contribute to the notion of coherence. |
Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent work in task-independent graph semantic parsing has shifted from symbolic approaches to neural models, showing strong performance on different types of meaning representations. |
| Approach: | They propose a framework that incorporates prior knowledge from a symbolic parser into a decision criterion for beam search to address these limitations. |
| Outcome: | The proposed framework improves on the in-distribution test set but degrades significantly on long-tail situations while the symbolic parser performs more robustly. |
Understanding Learning Dynamics Of Language Models with SVCCA (N19-1)
Copied to clipboard
| Challenge: | a new study shows that neural models implicitly encode linguistic features . but no research shows how these encodings arise as the models are trained . |
| Approach: | They propose a method that compares learning across time and across models using annotated data. |
| Outcome: | The proposed method compares learned representations across time and across models without evaluation on annotated data. |
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set (D19-1)
Copied to clipboard
| Challenge: | Using development sets for low-resource training is often more effective . however, some studies show that early stopping can overestimate performance . |
| Approach: | They find that early stopping on a development set is more effective than using all available data for training. |
| Outcome: | The proposed model overestimates accuracy over languages and tasks by 1.4% compared to a more realistic set of training epochs. |
CoNLL#: Fine-grained Error Analysis and a Corrected Test Set for CoNLL-03 English (2024.lrec-main)
Copied to clipboard
| Challenge: | a glass ceiling for named entity recognition systems has been suggested for 2021 . however, the performance of the most popular NER benchmarks has plateaued since then . we investigate what NER models are still struggling with . |
| Approach: | They perform a fine-grained evaluation of the model outputs by adding document annotations to the CoNLL-03 English dataset to identify lingering errors. |
| Outcome: | The proposed model is able to correct errors and guide future work. |
Attending to Long-Distance Document Context for Sequence Labeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | UC Berkeley researchers develop a method for incorporating global context in long documents . many of the main datasets used in NLP are comprised of relatively short documents - english OntoNotes contains 223 tokens . |
| Approach: | They propose a method for incorporating global context in long documents . they use multiple mentions of the same word type to generate a representation for each token . |
| Outcome: | The proposed model performs better at recognizing entities with high TF-IDF scores than parametric models lacking context. |
Decomposable Neural Paraphrase Generation (P19-1)
Copied to clipboard
| Challenge: | Existing models learn to generate paraphrases by mapping a sequence to another, with each word processed and generated in a uniform way. |
| Approach: | They propose a Transformer-based model that can learn and generate paraphrases at different levels of granularity in a disentangled way. |
| Outcome: | The proposed model achieves competitive in-domain performance compared to state-of-the-art models and significantly better performance when adapting to a new domain. |
Enhancing Automatic Readability Assessment with Pre-training and Soft Labels for Ordinal Regression (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models do not exploit ordinal nature of difficulty grades and make little effort for initialization to facilitate fine-tuning. |
| Approach: | They propose a readability assessment task that assigns a difficulty grade to a text . they use ordinal regression and pairwise relative text difficulty to train the model . |
| Outcome: | The proposed model outperforms competitive neural models and statistical classifiers on most datasets. |
Automatic Post-Editing of Machine Translation: A Neural Programmer-Interpreter Approach (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to inducing APE have suffered from over-correction, where the APE system tends to keep the machine translated text without any modification. |
| Approach: | They propose a neural programmer-interpreter approach to automated post-editing (APE) that mimics human perform post- editing using discrete edit operations . their model outperforms previous neural models for inducing PE programs on the WMT17 APE task for German-English up to +1 BLEU score and -0.7 TER scores. |
| Outcome: | The proposed model outperforms previous neural models for inducing PE programs on the WMT17 APE task for German-English up to +1 BLEU score and -0.7 TER scores. |
Learning to Generate Rules for Realistic Few-Shot Relation Classification: An Encoder-Decoder Approach (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a new approach to relation classification is proposed to use data-driven approaches to perform fewshot tasks with limited training data. |
| Approach: | They propose a neuro-symbolic approach for realistic few-shot relation classification via rules . they propose to generate rules that can be used to extract relations using custom T5-style models . |
| Outcome: | The proposed approach is interpretable and pliable and outperforms the state-of-the-art on TACRED and NYT29 benchmarks while maintaining pliability. |
Semantic Representation for Dialogue Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models for dialogue modeling lack ability to represent core semantics, such as ignoring important entities. |
| Approach: | They develop an algorithm to construct dialogue-level AMR graphs from sentence-level data and explore two ways to incorporate AMRs into dialogue modeling. |
| Outcome: | The proposed model is superior to existing models on dialogue understanding and response generation tasks. |
DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion (N19-1)
Copied to clipboard
| Challenge: | Existing datasets for sentence fusion are small and insufficient for training modern neural models. |
| Approach: | They propose a method for automatically-generating fusion examples from raw text . they apply their method to Wikipedia and Sports articles to generate fusion models . |
| Outcome: | The proposed method improves performance on WebSplit when viewed as a sentence fusion task. |
Legal Case Retrieval: A Survey of the State of the Art (2024.acl-long)
Copied to clipboard
| Challenge: | Recent years have seen increasing attention on Legal Case Retrieval (LCR) this task involves retrieving cases from a legal database of historical cases that are similar to a given query case. |
| Approach: | They present a survey of the major milestones made in legal case retrieval research . they seek to understand the datasets and recent neural models and their performances . |
| Outcome: | The proposed task is based on a dataset of historical cases similar to a given query case. |
Automatic Poetry Generation with Mutual Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | Existing models for automatic poetry generation are based on maximum likelihood estimation (MLE) MLE-based models tend to remember common patterns of the poetry corpus, which results in loss-evaluation mismatch. |
| Approach: | They propose to model the criteria and use them as explicit rewards to guide gradient update by reinforcement learning to motivate the model to pursue higher scores. |
| Outcome: | The proposed model outperforms the current state-of-the-art model and improves on Chinese poetry. |
On-device Structured and Context Partitioned Projection Networks (P19-1)
Copied to clipboard
| Challenge: | A challenge in on-device text classification is to build highly accurate models that fit in small memory footprint and have low latency. |
| Approach: | They propose an on-device neural network which learns compact projection vectors from raw text using structured and context-dependent partition projections. |
| Outcome: | The proposed model outperforms baseline models and surpasses RNN, CNN and BiLSTM models on dialog act and intent prediction. |
GLUCOSE: GeneraLized and COntextualized Story Explanations (2020.emnlp-main)
Copied to clipboard
Nasrin Mostafazadeh, Aditya Kalyanpur, Lori Moon, David Buchanan, Lauren Berkowitz, Or Biran, Jennifer Chu-Carroll
| Challenge: | Existing knowledge resources and pretrained language models do not include or readily predict GLUCOSE’s rich inferential content. |
| Approach: | They propose a platform for crowdsourcing GLUCOSE data at scale that uses semi-structured templates to elicit causal explanations. |
| Outcome: | The proposed model can be trained on human-readable stories and build similar models on unseen stories. |
PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Logical table-to-text generation requires models to derive logical-level facts from table records via logical inference. |
| Approach: | They propose a pretrained logical form generator framework to improve generation fidelity . they use a dataset to test the logical inference accuracy of the framework . |
| Outcome: | The proposed framework outperforms baselines on LOGICNLG and CONTLOG on two benchmarks. |
Structural Supervision Improves Few-Shot Learning and Syntactic Generalization in Neural Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have not investigated the relationship between a token's frequency in the training corpus and syntactic properties models learn about it. |
| Approach: | They develop controlled experiments that probe models’ syntactic nominal number and verbal argument structure generalizations for tokens seen as few as two times during training. |
| Outcome: | The proposed models can make syntactic generalizations for tokens seen as few as two times during training and transfer them to transformed contexts. |
Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis (D19-1)
Copied to clipboard
| Challenge: | despite its practical advantages, transductive learning is underexplored in natural language processing . despite the simplicity of the technique, it is understudied in natural languages . |
| Approach: | They conduct an empirical study of transductive learning for neural models . they fine-tune language models on an unlabeled test set to obtain test-set-specific word representations. |
| Outcome: | The proposed method improves state-of-the-art neural models in syntactic and semantic tasks. |
Generating Textual Explanations for Machine Learning Models Performance: A Table-to-Text Task (2022.lrec-1)
Copied to clipboard
| Challenge: | Numerical tables are widely used to communicate or report the classification performance of machine learning models with respect to a set of evaluation metrics. |
| Approach: | They propose a task where neural models are trained to generate textual explanations based on the metrics’ scores reported in numerical tables. |
| Outcome: | The proposed model outperforms existing methods and can be used to explain the performance of ML models. |
Non-neural Models Matter: a Re-evaluation of Neural Referring Expression Generation Systems (2022.acl-long)
Copied to clipboard
| Challenge: | In recent years, neural models have outperformed rule-based and classic approaches in NLG. |
| Approach: | They evaluate two English datasets and evaluate their performance using automatic and human evaluations. |
| Outcome: | The proposed model outperforms rule-based and classic approaches on two English datasets and is compared with human-based models. |
Lexicon Learning for Few Shot Sequence Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Past work has shown that many failures of systematic generalization arise from neural models’ inability to disentangle lexical phenomena from syntactic ones. |
| Approach: | They propose a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules. |
| Outcome: | The proposed model improves generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation. |
Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations (2020.acl-main)
Copied to clipboard
| Challenge: | a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions. |
| Approach: | They propose a framework for sanity checking models against inconsistent explanations . they apply the framework to a state-of-the-art neural natural language inference model . |
| Outcome: | The proposed framework can generate inconsistent explanations on a state-of-the-art model . it also addresses the problem of adversarial attacks with full target sequences . |
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing studies on DS-based relation extraction (RE) methods focus on handling label noise, but other factors may have been overlooked. |
| Approach: | They propose a method to automatically adjust DS-RE models to a shifted label distribution problem . they find this problem exists in real-world DS datasets and can be overcome . |
| Outcome: | The proposed method achieves consistent performance gains on DS-trained models with an up to 23% relative F1 improvement, which verifies their assumptions. |
A Framework for Adapting Pre-Trained Language Models to Knowledge Graph Completion (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has demonstrated that entity representations can be extracted from pre-trained language models to develop knowledge graph completion models that are more robust to the naturally occurring sparsity found in knowledge graphs. |
| Approach: | They propose unsupervised and supervised methods to extract more informative representations from pre-trained language models to develop knowledge graph completion models. |
| Outcome: | The proposed model outperforms recent neural models in terms of performance and unsupervised processing methods. |
Combining Unsupervised Pre-training and Annotator Rationales to Improve Low-shot Text Classification (D19-1)
Copied to clipboard
| Challenge: | supervised learning models perform poorly at low-shot tasks for which little labeled data is available for training. |
| Approach: | They propose to combine a bag-of-words embedding approach and a context-aware method to improve low-shot text classification. |
| Outcome: | The proposed method improves low-shot text classification with pre-training and rationales . the simple bag-of-words approach is the clear top performer when there are few training instances or less . |
MRF-Chat: Improving Dialogue with Markov Random Fields (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deep learning for open-domain dialogue include training end-to-end models to learn various conversational features like emotional content of response, symbolic transitions of dialogue contexts and persona of the agent and the user, among others. |
| Approach: | They propose a probabilistic approach using Markov Random Fields to augment existing deep-learning methods for improved next utterance prediction. |
| Outcome: | The proposed approach significantly improves the performance of existing state-of-the-art retrieval models for open-domain conversational agents. |
Towards Explainability in Legal Outcome Prediction Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Current legal outcome prediction models do not explain their reasoning in the real world, but human legal actors need to understand the model’s decisions. |
| Approach: | They propose a method for identifying the precedent employed by legal outcome prediction models and a taxonomy of legal precedent to compare human judges and neural models. |
| Outcome: | The proposed model learns to predict outcomes reasonably well, but its use of precedent is unlike that of human judges. |
Adversarial Concept Erasure in Kernel Space (2022.emnlp-main)
Copied to clipboard
| Challenge: | Large neural networks in NLP produce real-valued representations that encode the bit of human language that they were trained on. |
| Approach: | They propose a kernelization of the recently-proposed linear concept-removal objective and propose to remove linear subspaces from the representation space. |
| Outcome: | The proposed kernelization protects against the ability of nonlinear adversaries to recover the concept. |
A Logic-Driven Framework for Consistency of Neural Models (D19-1)
Copied to clipboard
| Challenge: | Recent advances in NLP have improved performance on benchmarks such as GLUE . however, tracking performance on a leaderboard is not sufficient to characterize model quality . |
| Approach: | They propose a framework for constraining neural models using logic rules to regularize them away from inconsistency. |
| Outcome: | The proposed framework can be used on natural language inference and is compatible with off-the-shelf learning schemes without model redesign. |
Neural Grammatical Error Correction with Finite State Transducers (N19-1)
Copied to clipboard
| Challenge: | Language model based GEC (LM-GEC) is a promising alternative to SMT and neural sequence-to-sequence models. |
| Approach: | They propose to use finite state transducers to improve LM-GEC by rescoring with neural language models. |
| Outcome: | The proposed model outperforms the best published results on the CoNLL-2014 test set and achieves far better relative improvements over the baselines. |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
Sneaking Syntax into Transformer Language Models with Tree Regularization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for incorporating syntactic inductive biases into transformers are limited . we introduce auxiliary loss function that converts bracketing decisions into differentiable orthogonality constraints on vector hidden states. |
| Approach: | They propose to introduce syntactic inductive biases into transformer circuits through a structured regularizer. |
| Outcome: | The proposed approach could unlock more robust and data-efficient learning in transformer language models . it integrates seamlessly with the standard LM objective, requiring no architectural changes. |
Rethinking Complex Neural Network Architectures for Document Classification (N19-1)
Copied to clipboard
| Challenge: | Neural network models for many NLP tasks have grown increasingly complex in recent years . authors of recent papers question the necessity of such architectures and find them quite effective . |
| Approach: | They propose to use regularization techniques borrowed from language modeling to improve model accuracy . they find that a simple biLSTM architecture with appropriate regularization yields competitive results . |
| Outcome: | a simple biLSTM model outperforms the state-of-the-art on four benchmark datasets . authors say that improvements are not real, but are attributed to mundane reasons . |
DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to improve the robustness of text classification models are token-, sentence-, and hiddenlevel augmentation. |
| Approach: | They propose an interpolation-based data augmentation approach called DoubleMix to improve the robustness of text classification models by learning the “shifted” features in hidden space. |
| Outcome: | The proposed approach outperforms several popular methods on six text classification benchmark datasets and visual analysis shows that the model features are highly interpretable. |
Paraphrase Generation: A Survey of the State of the Art (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using neural models, paraphrase generation research has shifted to neural methods . a recent study focused on paraphrases, which are used in language understanding tasks . |
| Approach: | They propose to use neural methods to generate fluent, diverse paraphrases from a sentence . they propose to combine large pretrained language models with other mechanisms to generate more advanced paraphrase generation models. |
| Outcome: | This paper examines various approaches to paraphrase generation with a main focus on neural methods. |
Conundrums in Cross-Prompt Automated Essay Scoring: Making Sense of the State of the Art (2024.acl-long)
Copied to clipboard
| Challenge: | Automated essay scoring (AES) is a task of assigning a single score to an essay . authors abandon sophisticated neural architectures and develop a simple feature-based approach . |
| Approach: | a team of researchers develop a feature-based approach to cross-prompt automated essay scoring that adopts a simple neural architecture. |
| Outcome: | a new approach to cross-prompt automated essay scoring can achieve state-of-the-art results. |
Show Some Love to Your n-grams: A Bit of Progress and Stronger n-gram Language Modeling Baselines (N19-1)
Copied to clipboard
| Challenge: | Experimental results show that standalone n-gram models lend themselves as natural choices for resource-lean or morphologically rich languages. |
| Approach: | They run experiments on 50 languages covering all morphological language families to compare n-gram models with lstm models. |
| Outcome: | The proposed extension outperforms an lstm language model on 42 languages while its extension which explicitly injects linguistic knowledge outperformed the character-aware neural model on 8 languages. |
Rethinking Self-Supervision Objectives for Generalizable Coherence Modeling (2022.acl-long)
Copied to clipboard
| Challenge: | Prior work on text generation models focused on new architectures for permuted document tasks. |
| Approach: | They propose to use a basic model architecture to improve coherence evaluation of machine generated text. |
| Outcome: | The proposed model improves on a task-independent test set and shows significant improvements in coherence evaluations of downstream tasks. |
Operation-guided Neural Networks for High Fidelity Data-To-Text Generation (D18-1)
Copied to clipboard
| Challenge: | Recent neural models for data-to-text generation generate descriptions that are not consistent with structured data. |
| Approach: | They propose a framework for data-to-text generation that uses symbolic operations to generate texts from structured data. |
| Outcome: | The proposed framework improves the fidelity of the generated texts to the input structured data. |
Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network (P19-1)
Copied to clipboard
| Challenge: | Existing methods for inter-sentence relation extraction do not fully exploit such dependencies. |
| Approach: | They propose a model that captures local and non-local dependencies using multi-instance learning and bi-affine pairwise scoring to predict the relation of an entity pair. |
| Outcome: | The proposed model performs comparable to state-of-the-art models on biochemistry datasets. |
Neural Legal Judgment Prediction in English (P19-1)
Copied to clipboard
| Challenge: | Recent work on legal judgment prediction has focused on Chinese, but only feature-based models have been considered in English. |
| Approach: | They propose a hierarchical version of BERT which bypasses BERT’s length limitation. |
| Outcome: | The proposed model outperforms existing models in binary violation classification, multi-label classification and case importance prediction. |
A Taxonomy of Empathetic Response Intents in Human Social Conversations (2020.coling-main)
Copied to clipboard
| Challenge: | Open-domain conversational agents or chatbots are becoming increasingly popular in the natural language processing community. |
| Approach: | They aim to combine dialogue act/intent modelling and neural response generation to produce a large-scale taxonomy for empathetic response intents. |
| Outcome: | The proposed method improves the response quality of chatbots and makes them more controllable and interpretable. |
NormBank: A Knowledge Bank of Situational Social Norms (2023.acl-long)
Copied to clipboard
| Challenge: | NormBank is a knowledge bank of 155k situational norms that can be used to ground flexible normative reasoning for interactive, assistive, and collaborative AI systems. |
| Approach: | They propose a new scheme for hierarchically organizing the seemingly unbounded social norms within a multivalent sociocultural frame. |
| Outcome: | The proposed framework can be used to ground flexible reasoning for interactive, assistive, and collaborative AI systems. |
Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in deep learning (DL) based APR models have demonstrated promising results by learning from large-scale bug-fix examples in a data-driven manner. |
| Approach: | They propose a meta-learning framework integrated with code pretrained language models to generate fixes for low-resource bugs with limited training samples. |
| Outcome: | The proposed framework learns better error-specific knowledge from high-resource bugs through efficient first-order meta-learning optimization, which allows for a faster adaptation to the target low-resourced bugs. |
Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity (D18-1)
Copied to clipboard
| Challenge: | Existing encoder-decoder models for open domain dialogue generate generic, uninformative, and non-coherent responses. |
| Approach: | They propose to introduce a measure of coherence as the GloVe embedding similarity between dialogue context and generated response to improve output diversity. |
| Outcome: | The proposed model improves on the OpenSubtitles corpus in terms of BLEU score and diversity metrics. |
Do RNN States Encode Abstract Phonological Alternations? (2021.naacl-main)
Copied to clipboard
| Challenge: | Sequence-to-sequence models have been successful in word formation tasks, but the opacity of the models makes it difficult to determine whether complex generalizations are learned or whether there is some level of generalization across related sound changes. |
| Approach: | They propose to train character-based sequence-to-sequence models for inflection of Finnish nouns into the genitive case, an inflation type which is encoded in the hidden states of an LSTM encoderdecoder trained to perform word infference. |
| Outcome: | The proposed models encode 17 different consonant gradation processes in a handful of dimensions in the RNN. |
Sequence-Level Mixed Sample Data Augmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language. |
| Approach: | They propose a data augmentation approach to encourage compositional behavior in neural networks . they propose to softly combine input/output sequences from the training set . |
| Outcome: | The proposed approach yields 1.0 BLEU improvement on translation datasets over baselines. |
A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to QA re-rank sentences use huge neural models or complex attentive architectures. |
| Approach: | They propose to exploit the intrinsic structure of the original rank with an effective word-relatedness encoder to achieve the highest accuracy among the cost-efficient models. |
| Outcome: | The proposed model takes 9.5 seconds to train on the WikiQA dataset, compared with 18 minutes required by a standard BERT-base fine-tuning. |
Improving Encoder by Auxiliary Supervision Tasks for Table-to-Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that our method not only has a good generalization but also outperforms previous methods on several metrics: BLEU, Content Selection, Content Ordering. |
| Approach: | They propose to build an entity graph from the input tables and introduce a reasoning module to perform reasoning on the graph. |
| Outcome: | The proposed method outperforms previous methods on several metrics: BLEU, Content Selection, Content Ordering. |
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on optimal decision-making are limited and only consider individuals in isolation. |
| Approach: | They propose a task and corpus for learning alignments between machine and human preferences based on a gamified voting game . |
| Outcome: | The proposed task and corpus show that current state-of-the-art NLP models still leave much room for improvement. |
NPRF: A Neural Pseudo Relevance Feedback Framework for Ad-hoc Information Retrieval (D18-1)
Copied to clipboard
| Challenge: | Existing neural IR models do not have a mechanism for treating expansion terms differently from the original query terms, making it difficult to combine them with existing PRF approaches. |
| Approach: | They propose an end-to-end neural PRF framework that can be used with existing neural IR models by embedding different neural models as building blocks. |
| Outcome: | Extensive experiments on two standard test collections confirm the effectiveness of the proposed framework in improving the performance of two state-of-the-art neural IR models. |
A Detailed Evaluation of Neural Sequence-to-Sequence Models for In-domain and Cross-domain Text Simplification (L18-1)
Copied to clipboard
| Challenge: | Xu et al., 2016) show that a simple neural architecture can be efficiently used for in-domain and cross-domain text simplification. |
| Approach: | They evaluate neural sequence-to-sequence models for text simplification on Wikipedia and Newsela datasets. |
| Outcome: | The proposed model can generalize across corpora and overcome challenges when tested on Wikipedia and Newsela datasets. |
Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Syntactic structures were deemed essential in natural language processing . but since the deep learning revolution, NLP has been dominated by neural models that do not consider syntactical structures in their design. |
| Approach: | They propose a model that models latent representations of words in a sentence . they use a conditional random field to model latent and dependency arcs . |
| Outcome: | The proposed model performs competitively to transformers on small to medium sized datasets. |
Social Bias Frames: Reasoning about Social and Power Implications of Language (2020.acl-main)
Copied to clipboard
| Challenge: | Language has enormous power to project social biases and reinforce stereotypes on people. |
| Approach: | They propose a new conceptual formalism that aims to model the pragmatic frames in which people project social biases and power differentials onto others. |
| Outcome: | The proposed model can model the pragmatic frames in which people project social biases and power differentials onto others. |
AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods and limitations for machine reading comprehension are insufficient for logical reasoning over text. |
| Approach: | They propose a neural-symbolic approach which passes messages over a graph representing logical relations between text units to predict an answer. |
| Outcome: | The proposed approach outperforms existing methods on ReClor and LogiQA. |
Multilingual Data Filtering using Synthetic Data from Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that effective filters can be created by utilising Large Language Models to synthetically label data, which is then used to train smaller neural models for filtering purposes. |
| Approach: | They extend this approach to languages beyond English to train neural models for filtering purposes. |
| Outcome: | The proposed approach is effective at filtering parallel text for translation quality and filtering for domain specificity. |
MBTI Personality Prediction for Fictional Characters Using Movie Scripts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing NLP models cannot predict character's personality types based on text classifications . character comprehension is the cornerstone of understanding stories in psychology and education. |
| Approach: | They propose a benchmark to predict movie character's MBTI or Big 5 personality types based on the narratives of the character. |
| Outcome: | The proposed model outperforms existing models in the task and is more accurate than random guesses. |
Lexicosyntactic Inference in Neural Models (D18-1)
Copied to clipboard
| Challenge: | lexicosyntactic inferences are triggered by surprising aspects of the syntactical context that a word occurs in. |
| Approach: | They build a factuality judgment dataset for English clause-embedding verbs in various syntactic contexts and use it to probe the behavior of current state-of-the-art neural systems. |
| Outcome: | The proposed model makes systematic errors that are visible through the lens of factuality prediction. |
Inducing Transformer’s Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural models lack systematic compositionality in learning symbolic structures . existing models lack this ability in learning symbols, despite being able to understand complex structures. |
| Approach: | They propose to use auxiliary sequence prediction tasks to train a Transformer model to understand compositional symbolic structures of input data. |
| Outcome: | The proposed model improves on the SCAN compositionality challenge, with only 418 (5%) training instances, and achieves 97.8% accuracy on the MCD1 split. |
Requirements and Motivations of Low-Resource Speech Synthesis for Language Revitalization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing research on speech synthesis systems for three Indigenous languages in Canada requires tens of hours of audio recordings to be trained. |
| Approach: | They build a system for three Indigenous languages spoken in Canada using 1 hour of training data and 10 hours of data to train low-resource models. |
| Outcome: | The proposed system can produce speech with comparable naturalness to a Tacotron2 model trained with 10 hours of data. |
Think about it! Improving defeasible reasoning by first modeling the question scenario. (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature suggests that a person forms a mental model of the problem scenario before answering questions. |
| Approach: | They propose to have a model first create a graph of relevant influences and leverage that graph as an additional input when answering a defeasible query. |
| Outcome: | The proposed model achieves state-of-the-art on three different defeasible reasoning datasets. |
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Using the counterfactual memorisation metric, we find that when training neural networks, models will memorise some inputs but not others. |
| Approach: | They use the counterfactual memorisation metric to build a resource that places 5M NMT datapoints on a memorisations-generalisation map and describe how the datapoint’s surface-level characteristics and a models’ per-datum training signals are predictive of memorising in NMT. |
| Outcome: | The proposed model places 5M NMT datapoints on a memorisation-generalisation map and shows how their surface-level characteristics and models’ per-datum training signals are predictive of memorising in NMT. |
Log-linear Guardedness and its Implications (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for erasing human-interpretable concepts from neural representations that assume linearity are not fully understood. |
| Approach: | They define linear guardedness as the inability of an adversary to predict the concept directly from the representation . they show that a log-linear model can be constructed that indirectly recovers the concept . |
| Outcome: | The proposed model can be constructed that indirectly recovers the erased concept in some cases. |
Towards Improving Neural Named Entity Recognition with Gazetteers (P19-1)
Copied to clipboard
| Challenge: | Currently, neural models for named entity recognition are based on data-driven models, with a strong emphasis on getting rid of the efforts for collecting external resources or designing hand-crafted features. |
| Approach: | They propose to use external gazetteers to efficiently access annotated data to generalize beyond the annotation of entities. |
| Outcome: | The proposed model can access external gazetteers while avoiding the effort to design hand-crafted features. |
Towards Semi-Supervised Learning for Deep Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing methods for semantic role labeling require an immense amount of semantic-role corpora and are therefore not suitable for low-resource languages or domains. |
| Approach: | They propose a semi-supervised method that outperforms the state-of-the-art on SRL . method explicitly enforcs syntactic constraints by augmenting the training objective with a syntastic-inconsistency loss component. |
| Outcome: | The proposed method outperforms the state-of-the-art on limited SRL training corpora on CoNLL-2012 English section. |
Interactive Machine Comprehension with Dynamic Knowledge Graphs (2021.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments on iSQuAD suggest that graph representations can result in significant performance improvements for RL agents. |
| Approach: | They propose to use graph representations to build and update graphs during information gathering and neural models to encode graph representation in RL agents. |
| Outcome: | Extensive experiments on iSQuAD show that graph representations can improve performance for RL agents. |
Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language? (2020.acl-main)
Copied to clipboard
| Challenge: | Despite the success of language models using neural networks, it remains unclear to what extent neural models have the generalization ability to perform inferences. |
| Approach: | They propose a method to evaluate whether neural models can learn systematicity of monotonicity inference in natural language. |
| Outcome: | The proposed method shows that neural models can perform inferences on unseen combinations of lexical and logical phenomena when syntactic structures are similar between training and test sets. |
Communication breakdown: On the low mutual intelligibility between human and neural captioning (2022.emnlp-main)
Copied to clipboard
| Challenge: | 0-shot performance of a neural caption-based image retriever is higher when fed captions from a human-produced caption generator . despite the fact that the caption generator does not take the set of distractor images into account, this performance is only marginally above chance level. |
| Approach: | They compare the 0-shot performance of a neural caption-based image retriever with captions from a human-produced captioner. |
| Outcome: | The proposed model performs better when given human-produced captions or neural captions . the best pre-trained model perform better when fed captions produced by an out-of-the-box model . |
A Chinese Corpus for Fine-grained Entity Typing (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for fine-grained entity typing are limited to English . a corpus of 4,800 mentions is manually labeled with free-form entity types . |
| Approach: | They propose a Chinese fine-grained entity typing task that uses crowdsourcing . they categorize each mention into 10 general types and use a large tag set to predict open set of types . |
| Outcome: | The proposed dataset contains 4,800 mentions manually labeled in Chinese . it also categorizes all the fine-grained types into 10 general types . |
Rationally Reappraising ATIS-based Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Recent state-of-the-art neural models have obtained F1-scores near 98% on the task of slot filling. |
| Approach: | They propose to fix annotation errors in ATIS and propose a rule-based grammar for slot filling that achieves a 95.82% F1 score. |
| Outcome: | The proposed grammar achieves a 95.82% F1-score on the ATIS domain. |
Detecting and Mitigating Hallucinations in Multilingual Summarisation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing faithfulness metrics for abstractive summarisation models focus on English . metric mFACT is best suited to detect hallucinations in cross-lingual transfer . |
| Approach: | They propose a method to evaluate the faithfulness of non-English summaries by translation-based transfer from multiple English faithfulness metrics. |
| Outcome: | The proposed method reduces hallucinations in cross-lingual transfer by weighing the loss of each training example by its faithfulness score. |
Learning Music Helps You Read: Using Transfer to Study Linguistic Structure in Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | et al., 2018a, 2018b) show that LSTMs can transfer from non-linguistic data to natural language models with different types of abstract structure. |
| Approach: | They propose to use transfer learning to analyze encoding of grammatical structure in neural language models. |
| Outcome: | The proposed method improves test performance on natural language despite no overlap in surface form or vocabulary. |
Transition-based Bubble Parsing: Improvements on Coordination Structure Prediction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing bubble representations encoding coordination boundaries and internal relationships are difficult to detect and parse . |
| Approach: | They propose a bubble parser to perform coordination structure identification and dependency-based syntactic analysis simultaneously. |
| Outcome: | The proposed bubble parser beats state-of-the-art approaches on coordination structure prediction . the proposed system is based on a GENIA corpus and a Penn treebank . |
An Empirical Investigation of Structured Output Modeling for Graph-based Neural Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | In the past few years, graph-based dependency parsers have led to impressive empirical successes on parsing accuracy. |
| Approach: | They propose to use a graph-based dependency parser to model global outputs. |
| Outcome: | The proposed model has been shown to perform better on sentence-level Complete Match metric compared with the previous model. |
Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies show remarkable success in end-to-end task-oriented dialog systems . however, most models rely on large training data, which is difficult to scalable for new domains with limited labeled data. |
| Approach: | They propose a shared-private network which exploits the relevance between the target domain and each domain. |
| Outcome: | The proposed model outperforms existing methods on multi-domain dialogue by 13.9% on average. |
Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for aspect-based sentiment analysis of review text use only a few keywords describing each aspect/sentiment without using any labeled examples. |
| Approach: | They propose a weakly-supervised approach for aspect-based sentiment analysis which uses only a few keywords describing each aspect/sentiment without using any labeled examples. |
| Outcome: | The proposed method generates quality joint topics and outperforms baselines significantly on benchmark datasets. |
Atomic Inference for NLI with Generated Facts as Atoms (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that can provide accurate explanations are not interpretable, i.e. they do not reflect the inner workings of the model. |
| Approach: | They propose to use LLM-generated facts as atoms to make interpretable models that can be used to make accurate predictions for each component part of an input. |
| Outcome: | The proposed method outperforms existing methods on natural language understanding tasks with a multi-stage fact generation process and a training regime that incorporates the facts. |
Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian (2020.coling-main)
Copied to clipboard
| Challenge: | MRC in other languages, including Russian, has not been well-addressed due to the lack of high-quality and large-scale datasets. |
| Approach: | They propose two Russian machine reading comprehension datasets that require reasoning over multiple sentences and commonsense knowledge to infer the answer. |
| Outcome: | The proposed datasets are more complex than the original ones for Russian . the results show that the proposed models are challenging for advanced models . |
Deep Neural Model Inspection and Comparison via Functional Neuron Pathways (P19-1)
Copied to clipboard
| Challenge: | a general method for the interpretation and comparison of neural models is proposed . we factor a complex neural model into its functional components . |
| Approach: | They propose a method that factored a complex neural model into its functional components . they use correlated task level and linguistic heuristics to identify correlated pathways . |
| Outcome: | The proposed method can be applied in a purely post-processing manner to understand neural models. |
Relation Extraction with Explanation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent studies focus on improving relation extraction accuracy but little is known about their explanability. |
| Approach: | They propose to automatically generate "distractor" sentences to augment the bags and train the model to ignore the distractors. |
| Outcome: | The proposed model improves extraction accuracy while also explanability. |
Revisiting DocRED - Addressing the False Negative Problem in Relation Extraction (2022.emnlp-main)
Copied to clipboard
| Challenge: | Using incomplete annotations, we find that false negative samples are prevalent in the DocRED dataset . we reannotate 4,053 documents in the dataset by adding the missed relation triples back to the original DocRED. |
| Approach: | They propose to re-annotate 4,053 documents in the document-level relation extraction dataset by adding missing relation triples back to the original DocRED. |
| Outcome: | The proposed dataset improves on the existing DocRED dataset by 13 F1 points. |
Representation Learning for Unseen Words by Bridging Subwords to Semantic Networks (2020.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings only include words that appeared in corpora where pre-tried embedds are learned. |
| Approach: | They propose a method to represent out-of-vocabulary words using subword information and knowledge. |
| Outcome: | The proposed method improves performance over baselines that only use subwords or knowledge to represent OOV words. |
Investigating Transformer-Guided Chaining for Interpretable Natural Logic Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Natural logic reasoning has received increasing attention lately, with several datasets and neural models proposed, though with limited success. |
| Approach: | They propose to iteratively perform 1-step neural inferences and chain together the results to generate a multi-step reasoning trace. |
| Outcome: | The proposed method has high accuracies on a multi-hop First-Order Logic (FOL) reasoning benchmark. |
Neural Conversational QA: Learning to Reason vs Exploiting Patterns (2020.emnlp-main)
Copied to clipboard
| Challenge: | Neural Conversational QA tasks such as ShARC require systems to answer questions based on the contents of a given passage. |
| Approach: | They propose to modify a data-set with fewer spurious patterns to exploit them . they also propose to build a heuristic-based program to exploit spurious clues . |
| Outcome: | The proposed program exploits spurious patterns in the ShARC dataset, compared to neural models. |
SummScreen: A Dataset for Abstractive Screenplay Summarization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing summarization datasets are constructed from various domains, such as news, and we characterize them using two entity-centric metrics. |
| Approach: | They propose to use a summarization dataset to evaluate TV series transcripts and recaps . they propose to employ two entity-centric metrics to evaluate the dataset . |
| Outcome: | The proposed model outperforms the existing model and its oracle counterparts in character overlap and accuracy. |
Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using labeled data, named entity recognition is labor-intensive, time-consuming and expensive. |
| Approach: | They propose a method which decomposes named entity into two parts: entity and context. |
| Outcome: | The proposed method improves the generalization ability of models under limited observational examples. |
Higher-order Comparisons of Sentence Encoder Representations (D19-1)
Copied to clipboard
| Challenge: | a technique developed by neuroscientists compares activity patterns of different measurement modalities . a recent study examined the correspondence between popular pretrained language encoders and human processing difficulty . |
| Approach: | They employ a technique to compare activity patterns of different measurement modalities . they establish a correspondence between widely-employed pretrained language encoders and human processing difficulty . |
| Outcome: | The proposed technique can be used to compare representational geometries of neural models . it does not require large training samples and is not prone to overfitting, authors say . |
Incremental Neural Lexical Coherence Modeling (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work on pretrained language models has led to significant improvements in a range of NLP tasks. |
| Approach: | They propose a coherence model which interprets sentences incrementally to capture lexical relations between them. |
| Outcome: | The proposed model interprets sentences incrementally to capture lexical relations between them. |
Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available. |
| Approach: | They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers . |
| Outcome: | The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models. |
Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation (P19-1)
Copied to clipboard
| Challenge: | Disentangling the content and style in the latent space is prevalent in text style transfer . recurrent neural networks (RNN) based encoder and decoder cannot deal with the long-term dependency . |
| Approach: | They propose a style transformer which disentangles style information in latent space . they propose encoding and decoding methods that disentangle style information . |
| Outcome: | The proposed method can achieve better style transfer and better content preservation. |
Counterfactuals of Counterfactuals: a back-translation-inspired approach to analyse counterfactual editors (2023.findings-acl)
Copied to clipboard
George Filandrianos, Edmund Dervakos, Orfeas Menis Mastromichalakis, Chrysoula Zerva, Giorgos Stamou
| Challenge: | Existing explanations for classifiers are counterfactual or contrastive . lack of universal ground truth for counterf actual edits hinders their evaluation . |
| Approach: | They propose a back translation-inspired evaluation methodology that utilises earlier outputs of the explainer as ground truth proxies to investigate the consistency of explainers. |
| Outcome: | The proposed method can provide valuable insights into the behaviour of predictor and explainer models and infer patterns that would otherwise be obscured. |
Exploring Neural Topic Modeling on a Classical Latin Corpus (2024.lrec-main)
Copied to clipboard
| Challenge: | Using topic modeling, it is possible to study Latin literature through methods and tools that support distant reading. |
| Approach: | They propose to use topic modeling to investigate thematic distribution of Latin corpus . they train, optimize and compare two neural models to evaluate which performs better . |
| Outcome: | The proposed model is compared with two neural models with a Classical Latin corpus and shows that it is coherent and interpretable. |
Causal Intervention Improves Implicit Sentiment Analysis (2022.coling-1)
Copied to clipboard
| Challenge: | Existing neural models struggle with implicit sentiment analysis because they latch onto spurious correlations, resulting in poor generalization and robustness. |
| Approach: | They propose a CausaL intervention model for implicit sEntiment ANalysis using instrumental variable to eliminate confounding causal effects and extract the pure causal effect between sentence and sentiment. |
| Outcome: | The proposed model extracts the pure causal effect between sentence and sentiment using instrumental variable. |
PARADE: A New Dataset for Paraphrase Identification Requiring Computer Science Domain Knowledge (2020.emnlp-main)
Copied to clipboard
| Challenge: | Paraphrase identification requires specialized domain knowledge to perform . state-of-the-art neural models and non-expert human annotators have poor performance on PARADE . |
| Approach: | They propose a benchmark dataset called PARADE for paraphrase identification that requires specialized domain knowledge. |
| Outcome: | The proposed dataset shows state-of-the-art models and non-expert human annotators have poor performance on PARADE. |
Cooking Up a Neural-based Model for Recipe Classification (2020.lrec-1)
Copied to clipboard
Elham Mohammadi, Nada Naji, Louis Marceau, Marc Queudot, Eric Charton, Leila Kosseim, Marie-Jean Meurs
| Challenge: | a dataset of cooking recipes in French is highly imbalanced due to collaborative nature of the dataset . authors propose a neural-based model to address the first task of the DEFT 2013 shared task . |
| Approach: | They propose a neural-based model to address the first task of the DEFT 2013 shared task . they use state-of-the-art embedding approaches and deep architectures to address imbalanced dataset . |
| Outcome: | The proposed model outperforms models that use only pretrained embeddings in micro and macro F1 scores. |
Abstract Text Summarization: A Low Resource Challenge (D19-1)
Copied to clipboard
| Challenge: | Existing datasets for multilingual text summarization are difficult to construct and lack of human knowledge and language processing abilities in computers makes text summaries a challenging task. |
| Approach: | They propose an iterative data augmentation approach which uses synthetic data along with the real summarization data for the German language. |
| Outcome: | The proposed system improves on the development and test sets on the German language text using the state-of-the-art “Transformer” model. |
NLProlog: Reasoning with Weak Unification for Question Answering in Natural Language (P19-1)
Copied to clipboard
| Challenge: | ambiguity in natural language is difficult to interpret due to large linguistic variability. |
| Approach: | They propose to use a Prolog prover to extend neural networks with logic programming to solve multi-hop reasoning tasks over natural language. |
| Outcome: | The proposed model outperforms baseline models on two question answering tasks and is competitive on the MedHop corpus. |
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection (2020.acl-main)
Copied to clipboard
| Challenge: | Recent pretrained contextual representations such as ELMo and BERT have led to impressive performance gains across a variety of NLP tasks, including semantic similarity detection. |
| Approach: | They propose a topic-informed BERT-based architecture for pairwise semantic similarity detection that adds topic information to pretrained contextual representations such as BERT. |
| Outcome: | The proposed model outperforms existing models on a variety of English language datasets and is highly performant. |
MTR: A Dataset Fusing Inductive, Deductive, and Defeasible Reasoning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for logical reasoning focus on monotonic logic and a single form of reasoning. |
| Approach: | They propose to use a dataset to study the human-like reasoning in machine reading comprehension. |
| Outcome: | The proposed dataset shows that state-of-the-art neural models perform noticeably worse than expected. |
Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have demonstrated the ability of neural language models to learn linguistic properties without direct supervision. |
| Approach: | They propose to use an artificial corpus generated by a PCFG to control the gender distribution in training data and determine under which conditions a model correctly captures gender information. |
| Outcome: | The proposed approach allows to control the gender distribution in training data and determine under which conditions a model correctly captures gender information or appears gender-biased. |
Context-based Virtual Adversarial Training for Text Classification with Noisy Labels (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies show that deep neural networks can memorize noisy labels with limited training time. |
| Approach: | They propose a virtual adversarial training method to prevent a classifier from overfitting to noisy labels. |
| Outcome: | The proposed method performs the adversarial training in the context rather than the inputs. |
AD-NLP: A Benchmark for Anomaly Detection in Natural Language Processing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Methods for Anomaly Detection in text have shown strong empirical results on ad-hoc anomaly setups that are usually made by downsampling some classes of a labeled dataset. |
| Approach: | They propose a unified benchmark for detecting various types of anomalies . they evaluate two strong shallow baselines and two current state-of-the-art neural approaches . |
| Outcome: | The proposed benchmarks provide insights into the knowledge the neural models are learning when performing the task. |
RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing intent detection approaches have relied on adaptively pre-training language models with large-scale datasets, yet the predominant cost of data collection may hinder their superiority. |
| Approach: | They propose a self-supervised framework dedicated to task-oriented dialogues which incorporates agent responses for pre-training in a two-stage manner. |
| Outcome: | The proposed framework outperforms the state-of-the-art frameworks for task-oriented dialogues on two real-world customer service datasets. |
MOCHA: A Multi-Task Training Approach for Coherent Text Generation from Cognitive Perspective (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent pre-trained language models have produced impressive results, but there is still a gap between human written texts and machine-generated outputs. |
| Approach: | They propose a multi-task training strategy for long text generation grounded on the cognitive theory of writing. |
| Outcome: | The proposed model achieves better results on three open-ended generation tasks than baselines. |
Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach (2024.emnlp-main)
Copied to clipboard
| Challenge: | Incorrect translation of rare words can severely degrade the accuracy of ST models . |
| Approach: | They propose a retrieval-and-demonstration approach to enhance rare word translation accuracy in ST models by incorporating retrieved examples into ST models. |
| Outcome: | The proposed approach outperforms other modalities and exhibits higher robustness to unseen speakers. |
“Covid vaccine is against Covid but Oxford vaccine is made at Oxford!” Semantic Interpretation of Proper Noun Compounds (2022.emnlp-main)
Copied to clipboard
| Challenge: | Proper noun compounds are used in short-form domains but are largely ignored in information-seeking applications. |
| Approach: | They propose to annotate a manually annotated dataset of 22.5K proper noun compounds . they use supervised learning to generate interpretations from the compounds based on target knowledge . |
| Outcome: | The proposed dataset is 60 times larger than prior noun compound datasets and includes non-compositional examples. |
Revisiting Pathologies of Neural Models under Input Reduction (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that modern neural models tend to be miscalibrated. |
| Approach: | They examine why models produce high-confidence predictions on inputs that appear nonsensical to humans . previous work suggested that models fail to assign low probabilities due to model overconfidence . |
| Outcome: | The proposed methods can be extended to reduce the number of examples but with the cost of miscalibration. |
Improving Chinese Word Segmentation with Wordhood Memory Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Contextual features are important in Chinese word segmentation (CWS) but it is difficult to integrate wordhood information into existing neural models. |
| Approach: | They propose a neural framework that integrates contextual wordhood information with several popular encoder-decoder combinations for Chinese word segmentation. |
| Outcome: | The proposed framework achieves state-of-the-art performance on five benchmark datasets. |
Structured Tuning for Semantic Role Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Recent neural network-driven semantic role labeling systems have shown impressive improvements in F1 scores. |
| Approach: | They propose a framework to tune models using softened constraints only at training time. |
| Outcome: | The proposed framework outperforms the baseline model with minimal training time and consistent improvements under low-resource scenarios. |
Scale-Invariant Infinite Hierarchical Topic Model (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing hierarchical topic models yield fragmented topics with overlapping themes whose expected probability becomes exponentially smaller along the depth of the tree. |
| Approach: | They propose a hierarchical infinite hierarchic topic model that adapts to topic creation to make expected topic probability decay considerably slower than existing models. |
| Outcome: | The proposed model has better topic uniqueness and hierarchical diversity than existing approaches. |
Evaluating the Factual Consistency of Abstractive Text Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | a weakly-supervised approach is needed to verify factual consistency . auxiliary span extraction tasks are useful for verifying factual consistent summaries . |
| Approach: | They propose a weakly-supervised approach for verifying factual consistency . they transfer the model to summaries generated by several neural models . |
| Outcome: | The proposed approach outperforms models trained with strong supervision on source documents and human evaluations. |
KnowMAN: Weakly Supervised Multinomial Adversarial Networks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to weakly supervised training lack labeled data . weakly-supervised training can result in heuristic but noisy labels . |
| Approach: | They propose a scheme that allows to control influence of signals associated with specific labeling functions. |
| Outcome: | The proposed scheme improves results compared to weakly supervised learning with a pre-trained transformer language model and a feature-based baseline. |
TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition (2020.acl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental information extraction task that focuses on extracting entities from a given text and classifying them using pre-defined categories. |
| Approach: | They propose to use “entity triggers” to facilitate label-efficient learning of NER models. |
| Outcome: | The proposed model is significantly more cost-effective than the traditional neural NER frameworks. |
ILCiteR: Evidence-grounded Interpretable Local Citation Recommendation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches for local citation recommendation map or translate a query to citation-worthy research papers. |
| Approach: | They propose a local citation recommendation task that uses latent evidence spans to recommend papers . proposed system retrieves ranked lists of evidence span and recommended paper pairs . |
| Outcome: | The proposed system retrieves ranked lists of evidence span and recommended paper pairs based on evidence from the existing literature. |
Enhancing Accessible Communication: from European Portuguese to Portuguese Sign Language (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing systems for translating European Portuguese into LGP glosses rely on hand-crafted rules . current systems rely only on toy examples, disregarding non-manual movements . |
| Approach: | They propose a corpora-driven rule-based machine translation system between European Portuguese and LGP glosses and two neural machine translation models. |
| Outcome: | The proposed system improves on existing translation systems and annotates a gold collection of the results. |
Domain Generalization via Switch Knowledge Distillation for Robust Review Representation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing models for review representations of unseen or anonymous users are limited by their in-domain nature. |
| Approach: | They propose to use in-domain user and product information to generalize reviews . they use switch knowledge distillation to learn review representations for unseen users . |
| Outcome: | The proposed model performs well for existing or anonymous unseen users. |
Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on bias in NLP only considers negative or pejorative language use. |
| Approach: | They propose a revised framing of bias in terms of intergroup social context and its effects on language output. |
| Outcome: | The proposed framework is based on a model of intergroup relationships in English language tweets. |
MTAdam: Automatic Balancing of Multiple Training Loss Terms (2021.emnlp-main)
Copied to clipboard
| Challenge: | In supervised and unsupervised learning, adding loss terms often leads to improved performance. |
| Approach: | They propose an algorithm that balances the gradient magnitude of loss terms across all layers . they use Adam to add loss terms to neural models, but add more terms as they are added . |
| Outcome: | The proposed method improves performance and improves training outcomes. |
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation (2023.acl-long)
Copied to clipboard
| Challenge: | Neural machine translation models induce a non-smooth representation space, which harms its generalization results. |
| Approach: | They propose a framework to smooth the representation space by adjusting neighbor representations with a small number of new parameters. |
| Outcome: | The proposed framework outperforms the state-of-the-art kNN-MT system with average gains of 1.99 COMET and 1.0 BLEU on four benchmark datasets. |
Learning Bidirectional Morphological Inflection like Humans (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent research has focused on whether neural models can acquire morphological inflection like humans. |
| Approach: | They propose to use a recurrent neural network with attention and the transformer to train a symbolic model under a human-like learning environment to evaluate their models. |
| Outcome: | The proposed models did not accurately inflect verbs in the same manner as humans in terms of morphological inflection direction. |
Learning Intrinsic Dimension via Information Bottleneck for Explainable Aspect-based Sentiment Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Gradient-based explanation methods are increasingly used to interpret neural models in natural language processing (NLP) however, in the context of Aspect-based Sentiment Analysis, only specific dimensions are pertinent. |
| Approach: | They propose a Gradient-based explanation framework that leverages an information bottleneck to refine word embeddings into a concise intrinsic dimension, maintaining essential features and omitting unrelated information. |
| Outcome: | The proposed framework improves both the models’ performance and explanations’ clarity by identifying sentiment-aware features. |
Retrieval-Augmented Retrieval: Large Language Models are Strong Zero-Shot Retriever (2024.findings-acl)
Copied to clipboard
Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Yibin Lei, Tianyi Zhou, Michael Blumenstein, Daxin Jiang
| Challenge: | Large-scale retrieval is indispensable in information-seeking tasks such as open-domain question answering and knowledgegrounded dialogue. |
| Approach: | They propose to use a large language model (LLM) to augment a query with its potential answers by prompting LLMs with a composition of the query and the query’s in-domain candidates. |
| Outcome: | The proposed method breaks brute-force combinations of retrievers with LLMs and lifts the performance of zero-shot retrieval to be very competitive on benchmark datasets. |
Aligning Predictive Uncertainty with Clarification Questions in Grounded Dialog (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Previous work bases the timing of questions on supervised models learned from interactions between humans. |
| Approach: | They propose to ground the need for questions in the acting agent's predictive uncertainty by using the T5 encoder-decoder architecture to solve a Minecraft Collaborative Building task. |
| Outcome: | The proposed model can detect ambiguous instructions and predict responses better than previous models. |
Neural Machine Translation between Low-Resource Languages with Synthetic Pivoting (2024.lrec-main)
Copied to clipboard
| Challenge: | Pivot-based neural machine translation systems overcome data scarcity by including a high-resource pivot language in the process of translating between low-resourced languages. |
| Approach: | They propose a novel approach to pivot-based translation in which pivot sentences are generated synthetically from both the source and target languages. |
| Outcome: | The proposed approach improves pivot-based systems translating between low-resource Southern African languages by up to 5.6 BLEU points after fine-tuning. |
OOVs in the Spotlight: How to Inflect Them? (2024.lrec-main)
Copied to clipboard
| Challenge: | Inflection is a process of word formation in which a base word form (lemma) is modified to express grammatical categories. |
| Approach: | They develop a retrograde model and two sequence-to-sequence models based on LSTM and Transformer. |
| Outcome: | The proposed systems outperform the existing systems on 9 out of 16 languages in the OOV evaluation. |
Social Orientation: A New Feature for Dialogue Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on social orientations in dialogues show they improve performance in low-resource settings. |
| Approach: | They propose to use social orientation tags to model dialogue outcomes . they introduce a new set of dialogue utterances machine-labeled with social orientation tag. |
| Outcome: | The proposed model improves on English and Chinese language benchmarks and shows that social orientation tags explain the outcomes of social interactions when used in neural models. |
From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing alignment benchmarks focus on sentence embeddings, but prior research has shown that neural models tend to induce a non-smooth representation space, which impact of semantic alignment evaluation on low-resource languages. |
| Approach: | They propose a novel cross-lingual alignment evaluation method based on the consistency of parallel sentences to assess model alignment. |
| Outcome: | The proposed method achieves a correlation of 0.9556 with downstream tasks performance and 0.8524 with transferability even with a small dataset. |
Massively Multilingual Joint Segmentation and Glossing (2026.acl-long)
Copied to clipboard
Michael Ginn, Lindia Tjuatja, Enora Rice, Ali Marashian, Maria Valentini, Jasmine Xu, Graham Neubig, Alexis Palmer
| Challenge: | Existing models generate morpheme-level glosses but assign them to whole words without predicting the actual morphological boundaries, making them less interpretable and therefore untrustworthy to human annotators. |
| Approach: | They propose to use neural networks to predict interlinear glosses and morphological segmentation from raw text. |
| Outcome: | The proposed model outperforms GlossLM on glossing and beats open-source models on segmentation, glossing, and alignment. |
PictoEduca: Building a Dataset for Spanish Text-to-Pictogram Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | PictoEduca is the first large-scale Spanish text-to-pictogram dataset for augmentative and alternative communication. |
| Approach: | They present PictoEduca, a large-scale Spanish text-to-pictogram dataset for augmentative and alternative communication. |
| Outcome: | The proposed dataset combines automatic annotation with targeted expert correction, supporting scalable and high-quality corpus construction. |